Associate Architect - Data - PySpark + Streaming

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.


If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Job Role : Associate Architect - Data

Experience : 6+ Years

Location : Mumbai/Bangalore/Trivandrum (Hybrid)

Must have skills:

• 6+ years of hands-on experience in building large-scale data pipelines.

• Strong experience with Spark Streaming, AWS Glue, and EMR for processing both real-time and batch workloads

• Proficiency in Pyspark/Python, including building Kafka producers for data ingestion

• Experience working with Confluent Kafka for streaming ingestion from on-premise sources.

• Solid understanding of AWS services including S3, Redshift, Glue, CloudWatch, and Secrets Manager

• Experience working with Medallion architecture and hybrid data destinations (e.g., Redshift and on-premise Oracle)

• Ability to implement monitoring dashboards and observability for pipelines using tools like CloudWatch or Datadog

• Strong SQL skills for validating data accuracy and developing job-level metrics

• Experience with building alerting mechanisms for pipeline failures and performance issues

• Strong collaboration and communication skills

• Proven ability to take ownership of technical deliverables, driving execution from design to deployment

• Experience in guiding junior developers or data engineers, providing technical direction and code reviews to uphold best practices

Good to have skills : 

• Experience with orchestration tools like Apache Airflow or AWS Step Functions

• Exposure to Terraform for infrastructure automation

• Familiarity with CI/CD pipelines for data workflows

• Prior experience in on-prem to cloud data migration

• AWS Solution Architect Certification

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!