Data Engineer - Fabric Pipelines / Orchestration
Tasks
- Build Fabric pipelines
- Build reusable patterns and automation frameworks
- Derive requirements from data in discovery sessions
- Design, develop, and maintain data pipelines
- Investigate and remediate data quality issues
- Investigate and remediate performance bottlenecks
- Investigate and remediate pipeline failures
- Manage code using GitHub
- Migrate Spark job definitions from Synapse to Fabric
- Monitor and support pipelines post go live
- Optimize pipelines for performance and scalability
- Refactor and clean up existing code
- Set up data orchestration
- Work within metadata driven architecture
Perks/Benefits
Skills/Tech-stack
Apache Spark | Azure Synapse | Data Quality | Data Warehousing | Data integration | Data orchestration | ETL | GitHub | Microsoft Fabric | Performance Tuning | SQL | Scala
Education
Bachelor of Engineering | Bachelor of Science | Bachelor of Technology
Roles
Related jobs
-
Senior-level Full TimePune, India1h ago
-
Senior Data Engineer - AWS INR 1500K-4000KAWS Glue | AWS Lambda | AWS S3 | Amazon EMR | Amazon RedshiftSenior-level Full TimeBangalore, India4h ago
-
Mid-level Full TimeBangalore - Karnataka - India Pune …5h ago
-
Mid-level Full TimeHyderabad, TS, 500032, IN6h ago
-
Principal Data Engineer INR 1500K-2500KAgile | Apache Airflow | Apache Kafka | Automated testing | CI/CDSenior-level Full TimeIN-MH-Pune6h ago
-
Senior-level Full TimeIN-MH-Pune6h ago
-
Agentic AI Engineer INR 1200K-2500KAgentic architecture | Distributed Systems | Docker | LLM | LanggraphMid-level Full TimeKarnataka, Bengaluru, India7h ago
-
Senior-level Full TimeBengaluru, India8h ago
-
Associate Software Engineer INR 1200K-2800KAgile | C# | Cloud Development | Containerization | DHTMLMid-level Full TimeKolkata, India8h ago
-
Software Engineer II - Big Data INR 1500K-2500KAWS | Ab Initio | Apache Spark | Batch Processing | Data MiningMid-level Full TimeHyderabad, Telangana, India9h ago
-
Software Engineer III - Big Data INR 2200K-4200KAWS | Ab Initio | Apache Spark | Application deployment | Batch ProcessingSenior-level Full TimeHyderabad, Telangana, India9h ago
-
Software Engineer II INR 1600K-2800KAgile | Application Resiliency | CI/CD | Cloud technologies | Data AnalysisMid-level Full TimeMumbai, Maharashtra, India10h ago
-
Full Stack Analytics Engineer INR 1500K-3000KAirflow | Amazon Redshift | Azure Synapse | CI/CD | Cloud ComputingSenior-level Full TimeMumbai, India10h ago
-
Full Stack Analytics Engineer INR 1500K-3000KAirflow | Amazon Redshift | Amazon Web Services | Azure | Azure SynapseSenior-level Full TimeMumbai, India10h ago
-
Senior Data Engineer INR 3000K-4000KAWS | Amazon Aurora | Apache Airflow | Apache Kafka | Apache SparkBike to work | Career progression and mentorship coaching | Educational assistance | Employee assistance programme | Employee discountSenior-level Full TimeMumbai, MH, India11h ago
-
Principal Engineer, Data Analytics Engineering [ 8+ years ] (Java/Python | Kafka | Kubernetes) INR 2000K-3584KAWS | Agile | Algorithms | Apache Spark | AutomationSenior-level Full TimeBengaluru, KA, India11h ago
-
Principal Data Engineer INR 1500K-3500KAPIs | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data FactorySenior-level Full TimeIndia, Telangana, Hyderabad11h ago
-
Senior-level Full TimeHyderabad, India12h ago
-
Senior Software Engineer INR 2800K-4200KAWS | Bash | CI/CD | CICS | CobolDevelopment opportunities | Reward and recognition | Volunteering | Wellness | Work-life balanceSenior-level Full TimeHyderabad, India12h ago
-
POD Engineers (Jr) INR 1500K-2000KAWS S3 | Amazon Web Services | Apache Kafka | Apache Spark | AzureIn office attendance as needed | Travel required 25 percentMid-level Full TimePune, MH, IN12h ago
-
Member of Technical Staff 3 INR 1500K-3000KAutomation | Cloud Architecture | Computing Services | Design Patterns | Integration TestingSenior-level Full TimeBengaluru, India12h ago
-
AI / ML Data Scientist (Across levels) INR 2520K-4580KAPI Development | AWS | Apache Kafka | Apache Spark | AzureSenior-level Full TimeBangalore, India12h ago
-
AI Engineer - FDE (Forward Deployed Engineer) INR 3242K-4500KAWS | Agent systems | Apache Spark | Azure | DspyRemote workSenior-level Full TimeRemote - India R13h ago
-
Software Engineer II- Data Engineer INR 1500K-2200KAWS | AWS EMR | AWS Glue | AWS Glue Catalog | AWS Glue ETLMid-level Full TimeMumbai, Maharashtra, India13h ago
-
Senior Machine learning Engineer INR 4000K-5500KAzure | Azure Blob | Azure Blob Storage | Azure ML | Blob StorageSenior-level Full Timebangalore, India13h ago