Data Engineer - Fabric Pipelines / Orchestration
Tasks
- Build Fabric pipelines
- Build reusable patterns and automation frameworks
- Derive requirements from data in discovery sessions
- Design, develop, and maintain data pipelines
- Investigate and remediate data quality issues
- Investigate and remediate performance bottlenecks
- Investigate and remediate pipeline failures
- Manage code using GitHub
- Migrate Spark job definitions from Synapse to Fabric
- Monitor and support pipelines post go live
- Optimize pipelines for performance and scalability
- Refactor and clean up existing code
- Set up data orchestration
- Work within metadata driven architecture
Perks/Benefits
Skills/Tech-stack
Apache Spark | Azure Synapse | Data Quality | Data Warehousing | Data integration | Data orchestration | ETL | GitHub | Microsoft Fabric | Performance Tuning | SQL | Scala
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology
Roles
Related jobs
-
Data Engineer INR 1500K-2500KApache Airflow | Apache Spark | Auto Loader | Azure Event | Azure Event HubSenior-level Full TimeIN-KA-Bangalore3h ago
-
Principal Architect AI Data Engineer INR 2000K-5000KAWS | Agent Orchestration | Azure | CI/CD | ContainersSenior-level Full TimeGurugram, Haryana, India4h ago
-
Architect AI Data Engineer INR 3000K-5000KAPI Integration | AWS | Agent Orchestration | Azure | Azure DatabricksSenior-level Full TimeNoida, Uttar Pradesh, India4h ago
-
Senior-level Full TimeGurugram, Haryana, India4h ago
-
Senior-level Full TimeNoida, Uttar Pradesh, India4h ago
-
Software Engineer III, AI/ML, Google Cloud INR 2500K-5000KAudio generation | Debugging | Distributed Computing | Documentation | GenAISenior-level Full TimeHyderabad, Telangana, India5h ago
-
Mid-level Full TimeBengaluru, Karnataka, India6h ago
-
Senior AWS Data Engineer INR 1500K-2500KAPI Gateway | AWS API | AWS API Gateway | AWS CloudFormation | AWS EMRSenior-level Full TimeIndia11h ago
-
Senior Analytics Engineer – Data Applications INR 2771K-4000KAirflow | Anomaly Detection | CI/CD | DBT | DashHybrid work modelSenior-level Full TimeBengaluru, INDIA, India14h ago
-
Forward Deployed Engineer INR 2040K-2800KAPI Integration | AWS | Apache Spark | Application development | AzureCustomer-facing role | Travel 20 percentSenior-level Full TimeRemote - India R23h ago
-
Senior-level Full TimeIndia1d ago
-
Quantum Prod Support INR 2755K-3500KBatch monitoring | Cron | ITIL | Incident Management | JiraHybrid work | Rotational shiftsSenior-level Full TimeIndia - Bengaluru; India - Hyderabad; …1d ago
-
Senior-level Full TimeBengaluru, Karnataka, India1d ago
-
Lead Software Engineer - Big Data INR 2500K-3500KApache Hive | Apache Impala | Apache Kafka | Apache Pig | Apache SparkSenior-level Full TimeMumbai, Maharashtra, India1d ago
-
Database Engineer INR 500K-1500KCI/CD | DB2 | Data Warehousing | Database performance | Database performance tuningMid-level Full TimePune, MH, India1d ago
-
Apache Airflow | Apache HBase | Apache Hive | Apache Impala | Apache KafkaSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Principal Engineer, Data Analytics Engineering [ 8+ years ] INR 2500K-3584KAWS | Algorithms | Azure | Big Data | CI/CDSenior-level Full TimeBengaluru, KA, India1d ago
-
Senior-level Full TimeIndia - Hyderabad1d ago
-
Senior Software Engineer - Full Stack & GenAI INR 3264K-5876KAWS | Angular | CI/CD | Docker | Event DrivenHybrid workSenior-level Full TimeBengaluru, KA, India1d ago
-
Agent Development | Agent Development Kit | Angular | Backend Development | C++Mid-level Full TimeBengaluru, Karnataka, India1d ago
-
Senior Engineer - Data Analyst INR 3000K-5012KAPI | Automation | Big Data | Big data analytics | CI/CDSenior-level Full TimeHadapsar, Pune, Maharashtra, IND, 4110131d ago
-
Machine Learning Engineer (MLE-3) INR 3000K-5000KAnomaly Detection | Apache Airflow | Boosting | Classification | Data VisualizationSenior-level Full TimeBengaluru, India1d ago
-
AWS | Advanced Excel | Agile Development | Apache Spark | AzureExecutive-level Full TimeKolkata, West Bengal, India R1d ago
-
Lead Software Engineer - Java and AI/ML INR 3500K-5000KAI Agents | Agile | Automated testing | CI/CD | Code modernizationSenior-level Full TimeMumbai, Maharashtra, India1d ago
-
Mid-level Full TimePune, Maharashtra, India1d ago