Data Engineer Senior Consultant
Tasks
- Build data ingestion workflows
- Build reusable parameterized pipelines
- Clean transform and aggregate datasets
- Collaborate with analysts architects and product owners
- Create master stitched data file
- Design ETL pipelines
- Develop rules engine
- Develop scalable physical data models
- Ensure audit traceability and business continuity
- Implement Azure data architecture
- Implement data quality checks profiling reconciliation and alerting
- Implement monitoring orchestration and error handling
- Implement versioning governance and validation for logic
- Integrate MDM sources and unify regional variations
- Manage safe rule changes
- Optimize pipeline performance for large datasets
- Participate in data discovery sessions
- Support agile scrum teams
- Translate analyst rules into engine components
Perks/Benefits
- N/A
Skills/Tech-stack
Agile Scrum | Azure | Azure Data | Azure Data Factory | Azure Synapse | Data Factory | Data Lake | Data Lake Storage | Data Modeling | Data orchestration | Databricks | Drools | ELT | ETL | MDM | Python | SQL
Education
N/A
Related jobs
-
Assistant Manager Machine Learning INR 1500K-1809KA/B | A/B Testing | AWS Glue | AWS Lambda | AWS SageMakerMid-level Full TimeIndia-Gurugram8h ago
-
Senior-level Full TimeKochi, India8h ago
-
Senior-level Full TimeIN-TN-Chennai8h ago
-
Associate Process Manager INR 1800K-2800KAgentic AI | Agile methodologies | Causal Inference | Cypher | Experimental DesignMid-level Full TimeMumbai, Maharashtra, India9h ago
-
Partner Engineer, Generative AI INR 2500K-3000KAI infrastructure | AWS | Azure | C plus plus | Cloud platformSenior-level Full TimeBangalore, India | Mumbai, India9h ago
-
Entry-level Full TimePune, India9h ago
-
Principal Consultant - Data & AI INR 2500K-3200KAI Services | Agile | Apache Kafka | Apache Spark | Artificial IntelligenceSenior-level Full TimeHyderabad, TS, IN; Bengaluru, KA, IN; …9h ago
-
AI Engineer INR 1340K-2400KAI Safety | AWS | AWS Lambda | Adversarial AI | Amazon BedrockDedicated learning week | Employee Share Savings Plan | Hybrid work | Learning and mentoring | Paid volunteer dayMid-level Full TimeBengaluru10h ago
-
Agentic AI/ Prompt Engineer INR 1800K-3000KCloud Platforms | Embeddings | LLMs | Langchain | LlamaindexMid-level Full TimeNoida, Uttar Pradesh, India10h ago
-
Digital Analytics/ Google Analytics / Adobe Analytics / Power BI / Azure Databricks/ SQL/ Python/PySpark DM-Analytics INR 1800K-2400KAzure Databricks | BigQuery | CI/CD | DAX | Data ModelingSenior-level Full TimeBangalore, KA, India10h ago
-
AWS | Automated testing | Azure | CI/CD | Data PipelinesMid-level Full TimeMumbai, Maharashtra, India10h ago
-
Senior Data Engineer INR 2500K-2829KApache Airflow | Apache Spark | Azure | Azure Data | Azure Data LakeEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India10h ago
-
Assistant Manager- Python Gen AI with GCP- Bangalore INR 1200K-1700KAI Agents | API Development | Agile methodologies | Application Performance Optimization | Application performanceWork from client office 5 days per weekMid-level Full TimeBangalore, Karnataka, India10h ago
-
Software Engineer III - Data Engineer INR 1200K-2000KAWS | Apache Airflow | Azure | Big Data | Data GovernanceSenior-level Full TimeMumbai, Maharashtra, India11h ago
-
Senior Associate -Applied AI ML -Digital INR 1050K-1496KAlgorithms | Apache Spark | Big Data | Classification | CompilersMid-level Full TimeBengaluru, Karnataka, India11h ago
-
Sr. SW Engineer - SDET - Automation, GenAI tooling, API Testing INR 2156K-3000KAI | AI Agents | API Testing | CI/CD | Cloud NativeHybrid workSenior-level Full TimeBengaluru, INDIA, India12h ago
-
IT Support Specialist 1 (Regional) INR 2000K-2000KAgile | Apache Spark | Azure Data | Azure Data Factory | Azure DevOpsSenior-level Full TimeBengaluru, KA, IN12h ago
-
Staff Software Engineer - Python, AI/ML, Cloud, GenAI INR 2000K-5000KAWS | Agent Orchestration | Azure | CI/CD | ContainerizationSenior-level Full TimeBengaluru, INDIA, India12h ago
-
AI Engineer- Associate Consultant INR 1500K-2000KAWS | Agent Orchestration | Autonomous Agents | Azure | ClassificationCareer growth | Collaborative work environment | Continuous learning | Inclusive work environmentMid-level Full TimeBangalore, Karnataka, India12h ago
-
Data Scientist Consultant INR 2500K-3000KAgent Orchestration | Agentic AI | Azure | Classification | ClusteringSenior-level Full TimeBangalore, Karnataka, India12h ago
-
Software Engineer III - AIML Platform Reliability INR 2500K-3500KAI Operations | AI Ops | AWS | Automation | Cloud ComputingSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
Executive-level Full TimeIndia12h ago
-
Software Engineer I - Python, AWS INR 2200K-3600KAWS | AWS IAM | Agile | Amazon CloudWatch | Amazon EMRSenior-level Full TimeMumbai, Maharashtra, India13h ago
-
Senior Database Engineer (3+ years, Aurora, Dynamo, Redis) INR 1244K-2200KAmazon Aurora | Amazon Aurora MySQL | Amazon Aurora PostgreSQL | Amazon Web Services | Aurora MySQL24x7 support rotation | Hybrid workSenior-level Full TimeBengaluru, INDIA, India13h ago
-
Senior Data Engineer - Azure Databricks INR 300K-500KACID | Apache Spark | Azure | Azure Data | Azure Data FactorySenior-level Full TimeHyderabad, TS, India13h ago