Data Engineering Lead (GCP, Bigquery, ETL, Prompt Engineering) Associate Director
Pune, Maharashtra, India
INR 1500K-2000K (estimate) Mid-level Full Time
Tasks
- Build production grade data models and analytical structures
- Define CI CD and automated testing standards
- Define non functional testing strategy and ensure resilience and recovery objectives
- Design batch and streaming data pipelines
- Develop resilient ELT ETL pipelines
- Ensure encryption lineage access control auditability and retention
- Establish metadata and data governance strategy
- Implement orchestration scheduling monitoring and alerting
- Lead data engineering team and coach engineers
- Leverage AI productivity tools for engineering effectiveness
- Manage incident and problem management processes
- Manage squad dependencies and third party delivery
- Optimize GCP spend and improve performance
- Own end-to-end data platform architecture
- Perform code reviews and release governance
- Track KPIs including DORA metrics
- Translate requirements into delivery plans
Perks/Benefits
Skills/Tech-stack
Access Control | Agile | Alerting | Apache Airflow | Auditability | Automated testing | BigQuery | CI/CD | Cloud KMS | Cloud Run | Cloud Storage | Cloud platform | Code review | Composer | DORA Metrics | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Vault | Dataflow | Dataproc | Dimensional Modeling | ELT | ETL | Encryption | FinOps | Functional testing | GKE | Google Cloud | Google Cloud Platform | IAM | Incident Management | KPIs | Lakehouse | Metadata Management | Monitoring | Network Security | Non-functional testing | Non-functional) | Observability | Orchestration | Problem Management | Pub/Sub | Release governance | Runbooks | Scheduling | VPC
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
SW Architect - AI Engineering INR 2500K-5000KAPI Development | AWS | Azure | Cloud platform | Cost OptimizationContinuous learning opportunities | Hybrid work opportunities | Supportive, inclusive work environment | Training programsSenior-level Full TimePune - Maharashtra - India1d ago
-
Lead Software Engineer - (SRE principles , AI tools, AWS) INR 3000K-6000KAIOps | API Gateway | AWS ECS | AWS Lambda | Amazon APISenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Consultant - Databricks Azure INR 1000K-2700KAccess Control | Azure | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeBangalore, Karnataka, India1d ago
-
Assistant Manager - Databricks Azure INR 1500K-2000KADLS | Amazon Kinesis | Apache Kafka | Apache Spark | Azure ADMid-level Full TimeBangalore, Karnataka, India1d ago
-
Consultant - Databricks AWS INR 1500K-2500KAWS Data | AWS Data Pipeline | AWS Glue | AWS Lake | AWS Lake FormationMid-level Full TimeBangalore, Karnataka, India1d ago
-
Senior-level Full TimeBangalore, Karnataka, India1d ago
-
Senior Data Analyst INR 2817K-4000KAlation | Analysis Services | Azure Analysis | Azure Analysis Services | Azure DataSenior-level Full TimeTelangana, India R1d ago
-
AWS Bedrock | Amazon Web Services | Anthropic Claude | Apache Airflow | Apache KafkaBonuses for technical content and knowledge sharing | Broadband reimbursement | Continuous learning and professional development | Inclusive global team environment | Medical insurance for parentsSenior-level Full TimeIndia1d ago
-
Data Science & Engineering Lead INR 3000K-4500KAPI Gateway | AWS Glue | AWS Lambda | Airbyte | Amazon EC2Senior-level Full TimeIndia - Remote R1d ago
-
AI Architect INR 2500K-5000KAWS SageMaker | Azure Machine Learning | Cloud Computing | Data Governance | Deep learningCollaborative working environment | Continuous learning | Global organisationSenior-level Full TimePune, MH, India1d ago
-
Lead Software Engineer INR 1500K-3000KAWS Cloud | Apache Airflow | Azure Data | Azure Data Factory | Data FactorySenior-level Full TimeBangalore, India1d ago
-
QA Lead - Manual, Automation & AI Testing INR 1500K-3000KAI Testing | API Integration | API Testing | Appium | Automation TestingSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Amazon Bedrock | Artificial Intelligence | BI | Batch Processing | CCPAEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysExecutive-level Full TimeBangalore, India1d ago
-
Data Engineering Developer- Senior Associate INR 1500K-2500KCI/CD | Data Modeling | Data Quality | Data Warehousing | Data pipelineEmployee networks | Flexible work/life support | Paid volunteer daysMid-level Full TimeHyderabad, India1d ago
-
Data Engineering Senior Analyst INR 3000K-3500KAWS | Agile | Azure | C# | CI/CDCross-team collaboration | Engage with multiple technological disciplines | Training and learning opportunitiesSenior-level Full TimeBengaluru, Karnātaka, India1d ago
-
Quality Engineering Lead – Data & Reporting Platforms INR 1600K-3000KAI Testing | API Testing | AWS Athena | Access Control | AgileSenior-level Full TimeLocation(s): Pune, Maharashtra, India1d ago
-
Engineering & Delivery Lead - Data Services INR 3672K-5000KAWS Athena | Apache Drill | Apache Flink | Apache Kafka | Apache RangerSenior-level Full TimeLocation(s): Pune, Maharashtra, India1d ago
-
Lead Fullstack Engineer- Advanced AI INR 2500K-4580KAccessibility | Agentic Systems | Application Security | CI/CD | Data integrationSenior-level Full TimeTower 02, Manyata Embassy Business Park, …1d ago
-
AI Engineering Manager INR 2000K-4000KAPI Integration | Automated testing | Claude API | Continuous Deployment | Continuous integrationMid-level Full TimeIndia, Bengaluru (Hybrid) R1d ago
-
Assistant Manager INR 1500K-2000KAccess Control | Agile | Airflow | Automated testing | Batch ProcessingMid-level Full TimePune, Maharashtra, India2d ago
-
Agile | Amazon Neptune | CI/CD | Data Governance | Data LineageFlexible working | Inclusive culture | Opportunities for growth | Professional developmentSenior-level Full TimeHyderabad, Telangana, India R2d ago
-
A/B | A/B Testing | API | Adaptive Models | ArbitrationContinuous professional development | Flexible working | Inclusive and diverse environment | Opportunities for career growthMid-level Full TimeHyderabad, Telangana, India R2d ago
-
Senior-level Full TimeIndia, Telangana, Hyderabad; India, Karnataka, Bangalore; …2d ago
-
Engineering Manager, Information Retrieval INR 3000K-4500KAgile | Artificial Intelligence | C# | C++ | Data CompressionSenior-level Full TimeBengaluru, Karnataka, India2d ago
-
Senior-level Full TimeBengaluru, Karnataka, India R2d ago