Data Engineer Senior Consultant
Tasks
- Assess Informatica workflows and translate to Python EL scripts and dbt models
- Build Airflow DAGs to orchestrate Python EL and dbt end to end
- Create dbt models with macros incremental strategies and snapshot patterns
- Create technical documentation runbooks and handover materials
- Design dbt transformation models using Informatica mapping logic
- Develop and maintain Python EL pipelines for high volume data loads
- Develop dbt native tests and Python unit tests for data quality
- Integrate dbt and Airflow with GitLab CI CD pipelines
- Perform code reviews and technical feedback
- Reconcile row counts between source systems and target warehouse
- Troubleshoot performance issues and data discrepancies during SIT and UAT
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | CI/CD | Cx Oracle | DBT Core | Data Pump | GitLab CI | GitLab CI/CD | Informatica PowerCenter | Microsoft SQL | Microsoft SQL Server | Oracle | Oracle Data | Oracle Data Pump | Pyodbc | Python | SQL | SQL Server | SQL Server BCP | SQLAlchemy | Unix
Education
N/A
Related jobs
-
Anthropic SDK | Embeddings | Fine Tuning | Git | GitHub ActionsCollaborative values led team | Exposure to global teams and customers | Generous holiday allowance | Hybrid work environment | Individual career developmentEntry-level Apprenticeship Full TimeWhitefield, Bangalore, India6h ago
-
Mid-level Full TimeMumbai, Maharashtra, India7h ago
-
Senior-level Full TimeMumbai, Maharashtra, India7h ago
-
Generative AI Engineer INR 1800K-3300KAI Studio | AWS SageMaker | Amazon Bedrock | Anthropic API | Azure AIGlobal consulting projects | Leadership trainingMid-level Full TimeMumbai, Maharashtra, India7h ago
-
Software Development Senior Associate INR 2000K-4500KAmazon Web Services | Artificial Intelligence | CI/CD | Data Analysis | DockerCareer growth opportunities | Certification support | Collaborative work environment | MentorshipSenior-level Full TimeChennai, TN, IN7h ago
-
Lead Data Engineer (OIC+Python) INR 2817K-4064KAPI Management | Apache Airflow | Cloud Composer | Cloud infrastructure | Cloud platformSenior-level Full TimeHyderabad, India8h ago
-
Sr. Software Engineer L4 INR 2500K-4000KAutomation | C plus plus | Cause analysis | Distributed Systems | File systemsSenior-level Full TimePune, MH, India10h ago
-
Lead Data Engineer INR 2000K-3000KApache Kafka | Apache Spark | Automated testing | CI/CD | ConcurrencySenior-level Full TimeBengaluru, KA, India11h ago
-
GenAI Engineer INR 1200K-4500KAI Studio | AKS | App Service | Autogen | AzureCareer growth programs | Flexible work arrangements | Flexible work hours | Generative AI Certifications Support | Remote work optionMid-level Full TimeGurgaon, IN12h ago
-
AI / ML Engineer INR 1200K-2500KAI Agents | Cloud Computing | Deep learning | Generative AI | Machine LearningMid-level Full TimeHyderabad, HDC3B, India14h ago
-
Senior Data Engineer - Python Job INR 2000K-3500KAmazon Web Services | CI/CD | Data Governance | Data Lineage | Data ModelingContinuous learning | Flexible work arrangements | Inclusive team environment | Stable employment | Team knowledge sharingSenior-level Full TimeBangalore, KA, IN14h ago
-
Principal Specialist- Data Science Customer Programs INR 3000K-4000KApache Spark | Data Engineering | Data Governance | Data Lineage | Data ModelingHybrid work environment | UK shift working hoursSenior-level Full TimeIN-KA-BENGALURU-NORTHGATE ~ Sy No 2/2 Venkatala … R14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Senior-level Full TimeBangalore, KA, IN14h ago
-
Mid-level Full TimeGurgaon, HR, IN, 12200214h ago
-
SEI - Data Migration Engineer (SWP) (Expanded Resources) INR 2400K-3380KApache Airflow | CI/CD | Cx Oracle | DBT Core | Data PumpSenior-level Full TimeBangalore, KA, IN14h ago
-
SEI - Data Migration Engineer (IMS) (Expanded Resources) INR 2520K-3880KApache Airflow | CI/CD | Cx Oracle | DBT | Data PumpSenior-level Full TimeBangalore, KA, IN14h ago
-
Specialist, Data Streaming Platform Engineer INR 2040K-2800KAirflow | Amazon Web Services | Apache Kafka | CI/CD | Confluent CloudDomestic relocation | Hybrid work arrangement | No travel requiredSenior-level Full TimeIND - Telangana - Hyderabad (Gachibowli …14h ago
-
SEI - Data Migration Engineer (IMS) (Expanded Resources) INR 2520K-3880KApache Airflow | Bulk Insert | CI/CD | Cx Oracle | DBT CoreSenior-level Full TimeBangalore, KA, IN14h ago
-
SEI - Data Migration Engineer (SWP) (Expanded Resources) INR 2400K-3380KApache Airflow | Bulk Insert | CI/CD | Cx Oracle | DBTSenior-level Full TimeBangalore, KA, IN14h ago
-
SEI - Data Migration Engineer (SWP) (Expanded Resources) INR 2400K-3380KApache Airflow | Bulk Insert | CI/CD | Cx Oracle | DBT CoreSenior-level Full TimeBangalore, KA, IN14h ago
-
Senior Specialist - Data Engineering INR 2800K-4000KAgile | Alteryx | Classification | Data Governance | Data MiningSenior-level Full TimeIND-CORP Gurugram-JBS- B2IT SEZ, India14h ago
-
Data Engineer II INR 1000K-2000KAmazon S3 | Apache Kafka | Apache Spark | CDC | CI/CDAutonomy | Inclusive and diverse environment | Learning stipend | Mentorship | Remote environmentMid-level Full TimeIndia - Remote R14h ago
-
None Full TimeIND-CORP Bengaluru-TDIM - PTT, India14h ago