Data Engineering Lead (GCP, Bigquery, ETL, Prompt Engineering) Associate Director
Pune, Maharashtra, India
INR 1500K-2000K (estimate) Mid-level Full Time
Tasks
- Build production grade data models and analytical structures
- Define CI CD and automated testing standards
- Define non functional testing strategy and ensure resilience and recovery objectives
- Design batch and streaming data pipelines
- Develop resilient ELT ETL pipelines
- Ensure encryption lineage access control auditability and retention
- Establish metadata and data governance strategy
- Implement orchestration scheduling monitoring and alerting
- Lead data engineering team and coach engineers
- Leverage AI productivity tools for engineering effectiveness
- Manage incident and problem management processes
- Manage squad dependencies and third party delivery
- Optimize GCP spend and improve performance
- Own end-to-end data platform architecture
- Perform code reviews and release governance
- Track KPIs including DORA metrics
- Translate requirements into delivery plans
Perks/Benefits
Skills/Tech-stack
Access Control | Agile | Alerting | Apache Airflow | Auditability | Automated testing | BigQuery | CI/CD | Cloud KMS | Cloud Run | Cloud Storage | Cloud platform | Code review | Composer | DORA Metrics | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Vault | Dataflow | Dataproc | Dimensional Modeling | ELT | ETL | Encryption | FinOps | Functional testing | GKE | Google Cloud | Google Cloud Platform | IAM | Incident Management | KPIs | Lakehouse | Metadata Management | Monitoring | Network Security | Non-functional testing | Non-functional) | Observability | Orchestration | Problem Management | Pub/Sub | Release governance | Runbooks | Scheduling | VPC
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Senior Solutions Architect INR 2000K-4131KAWS | Apache Cassandra | Apache Spark | Azure | Cloud ComputingHybrid work | Travel opportunitiesSenior-level Full TimeBengaluru, India R1d ago
-
Lead Software Engineer - Workday INR 1800K-2600KAPI Gateway | AWS CloudWatch | AWS Elastic Load Balancing | AWS Lambda | AWS VPCSenior-level Full TimeOff Embassy Golf Links Business Park, …1d ago
-
Senior-level Full TimeIndia - Bangalore Office1d ago
-
Software Site Reliability Engineer - AI & Data Platforms INR 2500K-4300KAWS | Apache Airflow | Apache Flink | Apache Kafka | Apache SparkMid-level Full TimeHyderabad1d ago
-
Principal Engineer, AI Architect INR 2000K-5000KAI Practices | API Integration | AWS | Agent Orchestration | AzureSenior-level Full TimeBengaluru, India1d ago
-
SW Architect - AI Engineering INR 2500K-5000KAPI Development | AWS | Azure | Cloud platform | Cost OptimizationContinuous learning opportunities | Hybrid work opportunities | Supportive, inclusive work environment | Training programsSenior-level Full TimePune - Maharashtra - India1d ago
-
Tech Lead INR 3000K-4500KApache Iceberg | Architecture Review | Audit Logging | Backend Engineering | Cloud NativeSenior-level Full TimeBengaluru, India2d ago
-
Lead Software Engineer - (SRE principles , AI tools, AWS) INR 3000K-6000KAIOps | API Gateway | AWS ECS | AWS Lambda | Amazon APISenior-level Full TimeBengaluru, Karnataka, India2d ago
-
Manager- Data Aggregation INR 1500K-2000KAmazon Redshift | Apache Airflow | Data Governance | Data Modeling | Data PipelinesMid-level Full TimePune2d ago
-
Consultant - Databricks Azure INR 1000K-2700KAccess Control | Azure | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeBangalore, Karnataka, India2d ago
-
Assistant Manager - Databricks Azure INR 1500K-2000KADLS | Amazon Kinesis | Apache Kafka | Apache Spark | Azure ADMid-level Full TimeBangalore, Karnataka, India2d ago
-
Technical Team Lead – LLM Systems (LangGraph & Bedrock) INR 5000K-6500KAWS Bedrock | Asynchronous programming | Event Driven | Event Driven Programming | LLM orchestrationEquity | Fast moving builder culture | Meaningful impactSenior-level Full TimeDelhi NCR2d ago
-
Consultant - Databricks AWS INR 1500K-2500KAWS Data | AWS Data Pipeline | AWS Glue | AWS Lake | AWS Lake FormationMid-level Full TimeBangalore, Karnataka, India2d ago
-
Senior-level Full TimeBangalore, Karnataka, India2d ago
-
Senior Data Analyst INR 2817K-4000KAlation | Analysis Services | Azure Analysis | Azure Analysis Services | Azure DataSenior-level Full TimeTelangana, India R2d ago
-
AWS Bedrock | Amazon Web Services | Anthropic Claude | Apache Airflow | Apache KafkaBonuses for technical content and knowledge sharing | Broadband reimbursement | Continuous learning and professional development | Inclusive global team environment | Medical insurance for parentsSenior-level Full TimeIndia2d ago
-
Data Science & Engineering Lead INR 3000K-4500KAPI Gateway | AWS Glue | AWS Lambda | Airbyte | Amazon EC2Senior-level Full TimeIndia - Remote R2d ago
-
AI Architect INR 2500K-5000KAWS SageMaker | Azure Machine Learning | Cloud Computing | Data Governance | Deep learningCollaborative working environment | Continuous learning | Global organisationSenior-level Full TimePune, MH, India2d ago
-
Lead Software Engineer INR 1500K-3000KAWS Cloud | Apache Airflow | Azure Data | Azure Data Factory | Data FactorySenior-level Full TimeBangalore, India2d ago
-
Lead Software Engineer INR 1500K-3000KAWS Cloud | Airflow | Apache Airflow | Azure Data | Azure Data FactorySenior-level Full TimeBangalore, India2d ago
-
QA Lead - Manual, Automation & AI Testing INR 1500K-3000KAI Testing | API Integration | API Testing | Appium | Automation TestingSenior-level Full TimeBengaluru, Karnataka, India2d ago
-
Amazon Bedrock | Artificial Intelligence | BI | Batch Processing | CCPAEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysExecutive-level Full TimeBangalore, India2d ago
-
Data Engineering Developer- Senior Associate INR 1500K-2500KCI/CD | Data Modeling | Data Quality | Data Warehousing | Data pipelineEmployee networks | Flexible work/life support | Paid volunteer daysMid-level Full TimeHyderabad, India2d ago
-
Data Engineering Senior Analyst INR 3000K-3500KAWS | Agile | Azure | C# | CI/CDCross-team collaboration | Engage with multiple technological disciplines | Training and learning opportunitiesSenior-level Full TimeBengaluru, Karnātaka, India2d ago
-
Quality Engineering Lead – Data & Reporting Platforms INR 1600K-3000KAI Testing | API Testing | AWS Athena | Access Control | AgileSenior-level Full TimeLocation(s): Pune, Maharashtra, India2d ago