Senior Data Engineer

About Us

Observe.AI is transforming customer service with AI agents that speak, think, and act like your best human agents—helping enterprises automate routine customer calls and workflows, support agents in real time, and uncover powerful insights from every interaction.

With Observe.AI, businesses boost automation, deliver faster, more consistent 24/7 service and build stronger customer loyalty.

Trusted by brands like Accolade, Prudential, Concentrix, Cox Automotive, and Included Health, Observe.AI is redefining how businesses connect with customers—driving better experiences and lasting relationships at every touchpoint.

The Opportunity

We are looking for a Senior Data Engineer with strong hands-on experience in building scalable data pipelines and real-time processing systems. You will be part of a high-impact team focused on modernizing our data architecture, enabling self-serve analytics, and delivering high-quality data products. This role is ideal for engineers who love solving complex data challenges, have a growth mindset, and are excited to work on both batch and streaming systems.

 What you’ll be doing:

• Build and maintain real-time and batch data pipelines using tools like Kafka, Spark, and Airflow.
• Contribute to the development of a scalable LakeHouse architecture using modern data formats such as Delta Lake, Hudi, or Iceberg.
• Optimize data ingestion and transformation workflows across cloud platforms (AWS, GCP, or Azure).
• Collaborate with Analytics and Product teams to deliver data models, marts, and dashboards that drive business insights.
• Support data quality, lineage, and observability using modern practices and tools.
• Participate in Agile processes (Sprint Planning, Reviews) and contribute to team knowledge sharing and documentation.
• Contribute to building data products for inbound (ingestion) and outbound (consumption) use cases across the organization.

Who you are:

• 5-8 years of experience in data engineering or backend systems with a focus on large-scale data pipelines.
• Hands-on experience with streaming platforms (e.g., Kafka) and distributed processing tools (e.g., Spark or Flink).
• Working knowledge of LakeHouse formats (Delta/Hudi/Iceberg) and columnar storage like Parquet.
• Proficient in building pipelines on AWS, GCP, or Azure using managed services and cloud-native tools.
• Experience in Airflow or similar orchestration platforms.
• Strong in data modeling and optimizing data warehouses like Redshift, BigQuery, or Snowflake.
• Exposure to real-time OLAP tools like ClickHouse, Druid, or Pinot.
• Familiarity with observability tools such as Grafana, Prometheus, or Loki.
• Some experience integrating data with MLOps tools like MLflow, SageMaker, or Kubeflow.
• Ability to work with Agile practices using JIRA, Confluence, and participating in engineering ceremonies.

Compensation, Benefits and Perks

• Excellent medical insurance options and free online doctor consultations
• Yearly privilege and sick leaves as per Karnataka S&E Act
• Generous holidays (National and Festive) recognition and parental leave policies
• Learning & Development fund to support your continuous learning journey and professional development
• Fun events to build culture across the organization
• Flexible benefit plans for tax exemptions (i.e. Meal card, PF, etc.)

Our Commitment to Inclusion and Belonging

Observe.AI is an Equal Employment Opportunity employer that proudly pursues and hires a diverse workforce. Observe AI does not make hiring or employment decisions on the basis of race, color, religion or religious belief, ethnic or national origin, nationality, sex, gender, gender identity, sexual orientation, disability, age, military or veteran status, or any other basis protected by applicable local, state, or federal laws or prohibited by Company policy. Observe.AI also strives for a healthy and safe workplace and strictly prohibits harassment of any kind.

We welcome all people. We celebrate diversity of all kinds and are committed to creating an inclusive culture built on a foundation of respect for all individuals. We seek to hire, develop, and retain talented people from all backgrounds. Individuals from non-traditional backgrounds, historically marginalized or underrepresented groups are strongly encouraged to apply.

If you are ambitious, make an impact wherever you go, and you're ready to shape the future of Observe.AI, we encourage you to apply. For more information, visit www.observe.ai.