Senior Data Engineering Lead
Primary location: Gurgaon, Haryana
INR 2520K-3500K (estimate) Senior-level Full Time
Tasks
- Build and optimize LLM RAG data pipelines including vector ingestion embeddings metadata enrichment
- Create data transformations for business requirements
- Detect data quality issues and implement root cause fixes
- Drive data governance security and responsible AI practices
- Implement ETL and ELT pipelines for relational and dimensional data
- Implement data integration for improved efficiency reliability value
- Implement modern data framework for business intelligence and advanced analytics
- Lead AI data engineering strategy and adoption observability and reusable datasets across domains
- Lead data acquisition from structured and semi structured source systems
- Load data marts and reporting aggregates
- Maintain data definitions documentation for governance and security
- Manage data audits and data quality remediation
- Monitor data pipelines for integrity and data quality issues
- Operationalize machine learning pipelines for scalable data workflows
- Prepare high level and detailed technical design documents
- Recommend data platform architecture changes with analytics teams
- Reduce technical debt and improve processes
- Support data engineering lifecycle research proof of concept design development testing deployment maintenance
- Troubleshoot and optimize production data systems
- Use DevOps tools for code versioning and deployment
- Utilize data management technologies for data engineering projects
Perks/Benefits
- N/A
Skills/Tech-stack
Access Control | Apache Spark | Azure Data | Azure Data Factory | Azure Databricks | Data Drift | Data Factory | Data Governance | Data Observability | Data Privacy | Data Quality | DataStage | Embeddings | GenAI | Informatica | LLM | Microsoft Azure | PLSQL | Prompt engineering | Python | RAG | REST API | SQL | SSIS | Scala | Semantic Search | TSQL | Vector Database
Education
N/A
Related jobs
-
Senior-level Full TimeIN-AP-Hyderabad4h ago
-
Quality Assurance (QA) - Lead - MiDAS INR 2000K-3300KAPI Testing | CI/CD | GitHub Actions | JMeter | LLM EvaluationHybrid workSenior-level Full Timebengaluru, India6h ago
-
Senior-level Full TimePune, Maharashtra, India9h ago
-
Lead AI/ML Engineer – Security AI & LLMs INR 3000K-5000KAPI Design | AWS SageMaker | Azure ML | Deep learning | DockerSenior-level Full TimePune, Maharashtra, India9h ago
-
Tech Lead - Java (Java, React, Kong) INR 1500K-3000KAPI Development | API Gateway | API Management | API Security | AgileSenior-level Full TimePune, IN, 41105712h ago
-
Lead Software Engineer - Python, Data, Cloud INR 3000K-4500KAPI Development | AWS Glue | Amazon Athena | Amazon Redshift | Amazon S3Senior-level Full TimeBengaluru, Karnataka, India12h ago
-
Senior Director, Panel Applications - Data Engineering INR 2520K-3380KAWS | Agile | Azure | Big Data | Cloud ComputingEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India12h ago
-
Gen AI Lead Developer INR 2500K-5000KAI Governance | Access rules | Computer Vision | DAMA | Data CatalogingSenior-level Full TimeIndia12h ago
-
Technical Lead-Azure Databricks INR 1500K-2700KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Senior-level Full TimeINDIA - BENGALURU - HP, IN17h ago
-
Lead Data & Analytics -Vice President-Software Engineer INR 3000K-4064KAPI Versioning | Azure | Core Java | Data Modeling | Data WarehousingSenior-level Full TimeCOMMERZ III, OBEROI GARDEN CITY, India17h ago
-
Lead Data Scientist- AML INR 3600K-5000KComputer Vision | Data Analysis | Data Structures | Data Structures and Algorithms | Data VisualizationSenior-level Full TimeTower 02, Manyata Embassy Business Park, …17h ago
-
Java Technical Lead- Career INR 2000K-3000KAWS | Airflow | Apache Beam | Apache Kafka | AzureHealthcare benefits | Hybrid work | Online learning platform | Paid time offSenior-level Full TimeIND-Trivandrum-Equifax Analytics-PEC, India17h ago
-
WFM Data Science Lead INR 2000K-3000KAnomaly Detection | Business Intelligence | Customer Experience | Dashboarding | Data PipelinesSenior-level Full TimeBengaluru, KA, IN23h ago
-
WFM Data Science Lead INR 2000K-3000KAmazon Redshift | Anomaly Detection | BPO | Contact center | Contact center operationsSenior-level Full TimeHyderabad, TS, IN23h ago
-
APIs | Agent systems | Architecture Design | Data Engineering | Data ModelingEquipment support | Flexible working hours | Gifts and rewards | Lunch allowance | Medical insuranceSenior-level Full TimeIndia R1d ago
-
AI Data Engineer III INR 3000K-4000KAgile | Amazon Web Services | Apache Airflow | CI/CD | DBTComprehensive insurance | Education assistance program | Fitness benefits | Flexible working hours | Health checkupsSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Big Data Lead INR 1500K-2500KAWS S3 | Amazon ECS | Amazon IAM | Apache Airflow | CI/CDClient facing communication | Team collaborationSenior-level Full TimeIndia1d ago
-
Lead Software Engineer -Python or Java, big data INR 3000K-4500KAWS | AWS Lambda | Agile | Amazon EMR | Amazon Elastic Compute ServiceSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Lead Software Engineer - AL/ML (Implementation Engineer) INR 4000K-5000KAWS | Azure | CI/CD | Distributed Systems | DjangoFlexible-hybrid work | Health insurance | Paid parental leave | Paid time offSenior-level Full TimeBengaluru1d ago
-
Assistant Director,Collections Analytics Lead ASP INR 2817K-4000KAML | Alteryx | BigQuery | Capacity Planning | Collections analyticsSenior-level Full TimeKolkata, West Bengal, India R1d ago
-
Senior Process Manager INR 2500K-5000KAI system architecture | AWS Bedrock | Autogen | Automation | Azure OpenAISenior-level Full TimePune, Maharashtra, India1d ago
-
Senior-level Full TimePune City,India1d ago
-
AI Governance | Airflow | Alerting | Anomaly Detection | Artifact RegistryMid-level Full TimePune, Maharashtra, India R1d ago
-
Lead Data Analyst INR 2817K-4064KAPI Integration | Dashboarding | Data Automation | Data Visualization | Data cleaningSenior-level Full TimeBengaluru1d ago
-
Lead, Health Analytics and AI Unit, Digital Health INR 3000K-4200KAdvanced Analytics | Artificial Intelligence | By Design | Data Engineering | Data GovernanceSenior-level Full TimeNew Delhi, DL, India1d ago