Senior Data Engineering Lead
Primary location: Gurgaon, Haryana
INR 2520K-3500K (estimate) Senior-level Full Time
Tasks
- Build and optimize LLM RAG data pipelines including vector ingestion embeddings metadata enrichment
- Create data transformations for business requirements
- Detect data quality issues and implement root cause fixes
- Drive data governance security and responsible AI practices
- Implement ETL and ELT pipelines for relational and dimensional data
- Implement data integration for improved efficiency reliability value
- Implement modern data framework for business intelligence and advanced analytics
- Lead AI data engineering strategy and adoption observability and reusable datasets across domains
- Lead data acquisition from structured and semi structured source systems
- Load data marts and reporting aggregates
- Maintain data definitions documentation for governance and security
- Manage data audits and data quality remediation
- Monitor data pipelines for integrity and data quality issues
- Operationalize machine learning pipelines for scalable data workflows
- Prepare high level and detailed technical design documents
- Recommend data platform architecture changes with analytics teams
- Reduce technical debt and improve processes
- Support data engineering lifecycle research proof of concept design development testing deployment maintenance
- Troubleshoot and optimize production data systems
- Use DevOps tools for code versioning and deployment
- Utilize data management technologies for data engineering projects
Perks/Benefits
- N/A
Skills/Tech-stack
Access Control | Apache Spark | Azure Data | Azure Data Factory | Azure Databricks | Data Drift | Data Factory | Data Governance | Data Observability | Data Privacy | Data Quality | DataStage | Embeddings | GenAI | Informatica | LLM | Microsoft Azure | PLSQL | Prompt engineering | Python | RAG | REST API | SQL | SSIS | Scala | Semantic Search | TSQL | Vector Database
Education
N/A
Related jobs
-
Lead AI/ML Engineer – Security AI & LLMs INR 3000K-5000KAPI Design | AWS SageMaker | Azure ML | Deep learning | DockerSenior-level Full TimePune, Maharashtra, India4h ago
-
Tech Lead - Java (Java, React, Kong) INR 1500K-3000KAPI Development | API Gateway | API Management | API Security | AgileSenior-level Full TimePune, IN, 4110577h ago
-
Senior Director, Panel Applications - Data Engineering INR 2520K-3380KAWS | Agile | Azure | Big Data | Cloud ComputingEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India7h ago
-
Technical Lead-Azure Databricks INR 1500K-2700KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Senior-level Full TimeINDIA - BENGALURU - HP, IN12h ago
-
APIs | Agent systems | Architecture Design | Data Engineering | Data ModelingEquipment support | Flexible working hours | Gifts and rewards | Lunch allowance | Medical insuranceSenior-level Full TimeIndia R1d ago
-
AI Data Engineer III INR 3000K-4000KAgile | Amazon Web Services | Apache Airflow | CI/CD | DBTComprehensive insurance | Education assistance program | Fitness benefits | Flexible working hours | Health checkupsSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Big Data Lead INR 1500K-2500KAWS S3 | Amazon ECS | Amazon IAM | Apache Airflow | CI/CDClient facing communication | Team collaborationSenior-level Full TimeIndia1d ago
-
Lead Software Engineer -Python or Java, big data INR 3000K-4500KAWS | AWS Lambda | Agile | Amazon EMR | Amazon Elastic Compute ServiceSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Lead Software Engineer - AL/ML (Implementation Engineer) INR 4000K-5000KAWS | Azure | CI/CD | Distributed Systems | DjangoFlexible-hybrid work | Health insurance | Paid parental leave | Paid time offSenior-level Full TimeBengaluru1d ago
-
Assistant Director,Collections Analytics Lead ASP INR 2817K-4000KAML | Alteryx | BigQuery | Capacity Planning | Collections analyticsSenior-level Full TimeKolkata, West Bengal, India R1d ago
-
Senior Process Manager INR 2500K-5000KAI system architecture | AWS Bedrock | Autogen | Automation | Azure OpenAISenior-level Full TimePune, Maharashtra, India1d ago
-
Senior-level Full TimePune City,India1d ago
-
AI Governance | Airflow | Alerting | Anomaly Detection | Artifact RegistryMid-level Full TimePune, Maharashtra, India R1d ago
-
Lead Data Analyst INR 2817K-4064KAPI Integration | Dashboarding | Data Automation | Data Visualization | Data cleaningSenior-level Full TimeBengaluru1d ago
-
Lead, Health Analytics and AI Unit, Digital Health INR 3000K-4200KAdvanced Analytics | Artificial Intelligence | By Design | Data Engineering | Data GovernanceSenior-level Full TimeNew Delhi, DL, India1d ago
-
Azure Databricks+ PySpark- Sr Technical Lead-Data Engg INR 1500K-2700KAzure Data | Azure Data Factory | Azure Data Lake | Azure Databricks | Data FactorySenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …1d ago
-
Data Analytics Specialist INR 2771K-4200KAPI | Agentic AI | Alteryx | Azure Data | Azure Data FactorySenior-level Full TimeGurgaon, HR, IN, 1220101d ago
-
AI Governance | Airflow | Alerting | Anomaly Detection | Artifact RegistryMid-level Full TimePune, MH, IN, 4110061d ago
-
IT Data & Analytics DevSecOps Manager INR 1000K-2000KAI Solutions | Agile | Amazon Web Services | Artificial Intelligence | AutomationOnsite work 3 days per weekMid-level Full TimeBangalore–Embassy Business Hub, India1d ago
-
Data Science Engagement Lead INR 2755K-4000KAWS | Agile | Artificial Intelligence | Azure | CI/CDCareer growth | Inclusive workplace | International collaboration | Training and development programmes | Work-life balanceSenior-level Full TimeSHELL CENTRE – CHENNAI, India1d ago
-
Data Analyst - Lead INR 2500K-4000KAdvanced Statistics | Algorithm Development | Artificial Intelligence | Campaign analytics | Customer SegmentationCareer mobility and development | Flexible work arrangements | Leadership programs | Training programs | Well-being programsSenior-level Full TimeFXE-MEISA/IND/GUXAT/GUXAT/Gurugram, India1d ago
-
Senior-level Full TimeIndia - Bangalore1d ago
-
APIs | Airflow | Amazon Redshift | Azure | BigQuerySenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India1d ago
-
Lead AI Engineer INR 4000K-5000KAPI Design | AWS | CI/CD | Containerization | DevOpsAdditional elected or voluntary benefits | Disability programs | Flexible working hours | Life insurance | Private medical careSenior-level Full TimeIN-MH-MUMBAI-HIRANANDANI BUSINESS PARK POWAI, India1d ago
-
Lead AI Engineer INR 4000K-5000KAPI Design | API Versioning | AWS | Artificial Intelligence | CI/CDAdditional elected or voluntary benefits | Disability programs | Flexible working hours | Life insurance | Private medical careSenior-level Full TimeIN-KA-BANGALORE-NEON BUILDING WEST TOWER, India1d ago