Data & Analytics Data Pipelines Lead
Tasks
- Coordinate data onboarding for new data sources
- Coordinate incident triage root cause analysis and resolution
- Define and maintain ETL ELT and data replication standards
- Define reconciliation exception handling and alerting frameworks
- Develop operational dashboards for platform performance
- Drive automation for data pipeline operational efficiency
- Ensure scalable secure high performing data integration architectures
- Establish data ingestion transformation orchestration delivery standards
- Establish monitor SLAs and operational metrics
- Implement data quality controls monitoring and validation
- Implement pipeline monitoring observability and alerting
- Improve reliability resiliency and recoverability
- Lead design development deployment support enterprise data pipelines
- Lead production support for data integration and analytics pipelines
- Maintain data catalog and metadata management adoption
- Maintain data lineage documentation
- Manage ServiceNow incidents service requests problem records and changes
- Manage batch near real time and streaming data ingestion
- Mentor data engineers and analysts
- Monitor pipeline performance and identify data integrity issues
- Oversee data movement across analytics lifecycle
- Serve as primary point of contact for pipeline operations
- Support disaster recovery and business continuity
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Alerting | Azure | Azure Data | Azure Data Factory | CI/CD | Cause analysis | Change Management | DBT | Data Catalog | Data Engineering | Data Factory | Data Governance | Data Lakes | Data Lineage | Data Mover | Data Observability | Data Pipelines | Data Quality | Data Reconciliation | Data Validation | Data Warehousing | Data integration | Databricks | DevOps | Dimensional Modeling | ELT | ETL | Fivetran | Google Cloud | ITSM | Incident Management | Informatica | Informatica PowerCenter | Kafka | Metadata Management | Monitoring | Oracle | Problem Management | Root Cause Analysis | Root cause | SSIS | ServiceNow | Snowflake | Synapse Analytics | Talend | Teradata | Teradata Data Mover
Related jobs
-
Lead Software Engineer - Pyspark, AWS, Databricks INR 3000K-5000KAWS | Agile | Apache Kafka | Application Resiliency | Automated testingSenior-level Full TimeBengaluru, Karnataka, India7h ago
-
Sr Lead Software Engineer - Agentic AI INR 3500K-5000KA2A SDK | APIs | AWS Lambda | Amazon EKS | Amazon S3Senior-level Full TimeBengaluru, Karnataka, India7h ago
-
Data & Analytics Engineer - AM INR 1500K-2500KApache Spark | Data Modeling | Data Warehousing | Databricks | Delta LakeSenior-level Full TimeBangalore, Karnataka, India7h ago
-
Apache Airflow | Apache Spark | BigQuery | CI/CD | Cloud ComposerMid-level Full TimePune, Maharashtra, India R8h ago
-
Lead Software Engineer - Lead Data Architect INR 3000K-5000KAWS | CI/CD | Cassandra | Data Governance | Data LineageSenior-level Full TimeMumbai, Maharashtra, India8h ago
-
Lead Data Engineer - Strategic Data Provisioning Data Engineer INR 3845K-5000KAI for Data | AI/ML | AWS | Agile | Apache SparkSenior-level Full TimeBengaluru, Karnataka, India8h ago
-
AI Assisted Development | APIs | AWS | AWS Lambda | AgileSenior-level Full TimeBengaluru, Karnataka, India11h ago
-
Lead Software Engineer INR 2500K-5000KAWS | Asynchronous programming | Authentication | Authorization | AutogenSenior-level Full TimeRemote, India R16h ago
-
Sr. Data Architect (Databricks) INR 1500K-2500KAWS | Apache Spark | Azure | Azure Data | Azure Data FactorySenior-level Full TimeGDC Financial District, India20h ago
-
Manager, Data Engineering, Smai INR 1500K-2028KAgentic AI | Apache NiFi | Apache Spark | Cloud Data | Cloud Data WarehouseMid-level Full TimeHyderabad - Phoenix Aquila, India20h ago
-
Lead Software Engineer - Specialist INR 2771K-3465KAWS | Airflow | Apache Beam | Atlassian Confluence | Atlassian JiraHealthcare benefits | Hybrid work | Online learning platform | Paid time offSenior-level Full TimeIND-Trivandrum-Equifax Analytics-PEC, India20h ago
-
Databrick Platform Admin/Data Engineering Engineer IV INR 1300K-2500KAWS | Agile | Apache Kafka | Apache Spark | AzureAccident insurance | Bereavement leave | Diversity, equity, and inclusion programs | Employee assistance program | Employee referral programMid-level Full TimeIND - Bengaluru, India20h ago
-
AI Lead and Automation Tester - Document Intelligence INR 2500K-5000KAI Workflow Orchestration | AI workflow | Automation Testing | Clinical Trial Compliance | Clinical trialSenior-level Full TimeIN-TN-Chennai1d ago
-
Lead Data Platform Engineer INR 2040K-3900KAPI Integration | AWS | AWS CloudWatch | AWS DataZone | AWS GlueEnhanced medical benefits | Family well-being benefits | Hybrid work | Paid time off | Wellbeing programsSenior-level Full TimeHyderabad, India1d ago
-
Senior Director, Panel Applications - Data Engineering INR 2520K-3380KAWS | Agile | Azure | Big Data | Cloud ComputingEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India1d ago
-
Data Science Lead INR 2817K-5000KAWS | Apache Airflow | Experiment design | Lookalike modeling | MLOpsCommunity volunteering | Financial advisory services | Group health coverage | Holidays | Hybrid work modelSenior-level Full TimeBengaluru R1d ago
-
AWS | Apache Spark | Business Intelligence | Data Architecture | Data GovernanceCareer growth | Continuous learning | Employee recognition | Global team collaboration | Inclusive workplaceSenior-level Full TimeIndia1d ago
-
Full Stack Tech Lead - Python + AI INR 2000K-4500KAWS Lambda | AWS SQS | Agentic development | Agile | AngularSenior-level Full TimeIndia1d ago
-
Applied AI ML Lead INR 4000K-5572KAI Agents | Data Governance | Data Management | Deep learning | Experiment trackingSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
IT - Advanced Analytics and Data Solutions Management INR 2500K-4000KAI machine learning | Acceptance Testing | Big Data | Business Intelligence | Cause analysisHealth and wellbeing programs | Personal and professional growth | Supportive work cultureMid-level Full TimeBangalore,IND, India1d ago
-
Director, Software Engineering, Self-service Operational Analytics and Reporting - ITC INR 3000K-6000KAgile | Alteryx | Apache Spark | CI/CD | DatabricksExecutive-level Full TimeINDIA TECHNOLOGY CENTER, India1d ago
-
Lead Generative AI Engineer INR 2500K-5000KAPI Integration | Batching | CI/CD | Containers | EmbeddingsDisability programs | Flexible working hours | Life insurance | Private medical care | Tailored financial programsSenior-level Full TimeIN-KA-BANGALORE-NEON BUILDING WEST TOWER, India1d ago
-
Data Science & AI Lead INR 3000K-5400KAI DevOps | API Integration | AWS | Algorithm Development | AzureIn person at least 3 days per week | Office basedSenior-level Full TimeBangalore–Embassy Business Hub, India1d ago
-
Data & AI Solutions manager/analyst INR 3380K-4720KAzure | C++ | Cause analysis | Data Compliance | Data GovernanceIn-person work at least 3 days per weekSenior-level Full TimeBangalore–Embassy Business Hub, India1d ago
-
Senior-level Full Time111442-IND-BENGALURU-INTL BLR BLK B3 PETUNIA, India1d ago