Group Lead - Data Quality Engineer
Bangalore, Karnataka, India
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Build Python backend services
- Build dashboards for data trust scores
- Build data quality and observability DataApps
- Define data quality KPIs
- Define data quality policies
- Detect volume and distribution anomalies
- Develop Streamlit dashboards and UIs
- Develop data observability capabilities
- Develop metadata driven quality monitoring
- Develop reusable Spark and Delta Lake frameworks
- Drive adoption of standardized data quality practices
- Ensure auditability traceability governance compliance
- Implement automated alerting
- Implement automated remediation workflows
- Implement data validation frameworks
- Integrate DQ checks into CI CD pipelines
- Integrate data quality checks in Azure pipelines
- Integrate monitoring with ADLS Gen2
- Monitor data freshness and SLAs
- Monitor schema drift and pipeline health
- Optimize quality validation performance
- Support audit and compliance reporting
- Validate data lineage and reliability
Perks/Benefits
- N/A
Skills/Tech-stack
ADLS Gen2 | Anomaly Detection | Apache Spark | Automated alerting | CI/CD | Data Freshness Monitoring | Data Lineage | Data Observability | Data Validation | Data freshness | Databricks | Delta Lake | Freshness Monitoring | Metadata Driven | Metadata Management | Metadata-driven architecture | Microsoft Azure | Python | REST API | SQL | Schema drift | Streamlit
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Amazon Web Services | Apache Airflow | Apache Kafka | Apache Spark | Cloud platformContract employment | Hybrid work | VIRTUAL interviewSenior-level ContractBangalore North, India2h ago
-
Data Engineer Architect. INR 1500K-2400KAPI Development | AWS | Agentic Engineering | Database optimization | FastAPISenior-level Full TimeIN-MH-Pune2h ago
-
Senior GenAI Developer INR 2500K-4800KAPI Gateway | AWS Bedrock | Agent systems | Agentic RAG | Amazon LambdaCertification programs | Global client projects | Health insurance | Internship opportunities | Language coursesSenior-level Full TimeAll Cities, India R4h ago
-
Senior-level Full TimePune, Maharashtra, India4h ago
-
Software Engineer II - Gen AI/ Agentic AI INR 1500K-2500KAWS | Amazon Elastic Kubernetes Service | Amazon SageMaker | Data Processing | Data integrationMid-level Full TimeHyderabad, Telangana, India6h ago
-
Senior-level Full TimeBengaluru, Karnataka, India6h ago
-
Senior Lead Software Engineer - Java, PySpark, and AWS INR 1800K-2400KAWS | AWS Glue | AWS Lake Formation | AWS Lambda | AWS Step FunctionsSenior-level Full TimeHyderabad, Telangana, India6h ago
-
AWS | Agile | Amazon Web Services | Apache Airflow | Application ResiliencySenior-level Full TimeBengaluru, Karnataka, India7h ago
-
Senior Software Engineer (Data Engineering, Python, SQL, Snowflake, Cloud and Devops) INR 2500K-3380KAWS | AWS Step Functions | Airbyte | Airflow | Apache HudiCareer development opportunities | Employee Assistance Program (EAP) | Financial wellness tools | Flexible working environment | LinkedIn LearningSenior-level Full TimeChennai, TN, India7h ago
-
AWS | Agile | Application Resiliency | CI/CD | DatabricksMid-level Full TimeHyderabad, Telangana, India7h ago
-
Azure Data & Fabric Developer / ETL pipeline & SQL ( 5 + yrs Assistant Manager - RNA INR 1500K-2000KAzure Blob | Azure Blob Storage | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeMumbai, MH, India8h ago
-
Data Science and Gen AI Developer-Associate INR 900K-2000KAutogen | CI/CD | Data Analysis | Data Visualization | ETLMid-level Full TimeBengaluru, Karnataka, India8h ago
-
Lead Software Engineer INR 1800K-2400KAWS | AWS Glue | AWS Identity | AWS Identity and Access Management | AWS Key Management ServiceSenior-level Full TimeMumbai, Maharashtra, India9h ago
-
Assistant Manager - SA2 (AI Hub - GDC) INR 1400K-2200KAWS | Agent Development | Apache Kafka | Autogen | AzureMid-level Full TimeHyderabad, Telangana, India9h ago
-
AWS | Agile | Amazon Web Services | Apache Airflow | CI/CDSenior-level Full TimeBengaluru, Karnataka, India9h ago
-
Senior Data Engineer INR 2800K-4000KAWS Elastic MapReduce | Agile | Apache Spark | Business Intelligence | Data WarehousingAnnual health check eye checkup dental screening OPD reimbursement | Employee assistance program | Free transportation | Leave policy | Medical insuranceSenior-level Full TimeBengaluru, Karnataka, India10h ago
-
Lead Software Engineer INR 3150K-4500KAWS EMR | AWS SNS | AWS SQS | Amazon Web Services | CI/CDCareer development opportunities | Comprehensive benefits | Flexible working arrangements | Work-life balanceSenior-level Full TimeMumbai, India11h ago
-
Applied Computer Vision Engineer - Data Driven Development INR 2500K-3630KAWS | Apache Spark | Azure | Computer Vision | Convolutional Neural NetworksSenior-level Full Timebengaluru, India12h ago
-
Data Engineer Azure - Senior INR 3000K-4000KApache Spark | Azure | Azure Data | Azure Data Factory | Azure Data LakeGlobal scale | Inclusive culture | Learning culture | Supportive teamSenior-level Full TimePune, MH, IN, 41101413h ago
-
Custom Software Engineer INR 1400K-1848KAirflow | CI/CD | Cause analysis | Change Management | Control-MRotational shifts | Stakeholder communicationMid-level Full TimeBengaluru, BDC7A, India15h ago
-
Mid-level Full TimePLOT NO-1, S.NO. 77, India15h ago
-
.NET | ASP.NET | Amazon Web Services | Angular | Apache AirflowFlexible work arrangements | Inclusive benefits | MentorshipSenior-level Full TimeKolkata DN 57, India15h ago
-
IN_Senior Associate_SCM_Data and Analytics_Advisory_Bangalore INR 2040K-3380KAgile | Amazon Web Services | Apache Airflow | Apache Hadoop | Azure DataFlexibility programs | Inclusive benefits | MentorshipSenior-level Full TimeBengaluru Millenia, India15h ago
-
Gen AI Engineer / Applied AI Engineer INR 1800K-2500KAI Agents | Cloud Run | Embeddings | Gemini | Generative AIHands on experience with Vertex AI and Gemini ecosystem | Opportunity to work on production systems | Work on challenging problemsMid-level Full TimeGurugram, Haryana, India15h ago
-
Senior PySpark Data Engineer INR 2000K-2800KApache Airflow | CI/CD | Cloudera | Data Modeling | Data WarehousingSenior-level Full TimePLOT NO-1, S.NO. 77, India15h ago