Associate Principal Engineer, Azure Data Factory Architect
Tasks
- Architect Azure Data Factory solutions
- Build Data Integration solutions
- Collaborate with stakeholders on integration requirements
- Define architecture standards and best practices
- Design ADF pipelines triggers datasets linked services
- Develop and optimize ADF Mapping Data Flows
- Ensure data reliability scalability and high availability
- Implement CI CD for ADF with Azure DevOps
- Implement ETL/ELT workflows
- Integrate ADF with Azure services
- Mentor data engineers and review code
- Monitor log and handle errors in ADF
- Optimize pipeline performance
- Perform root cause analysis
- Troubleshoot production issues
Perks/Benefits
- N/A
Skills/Tech-stack
ADF dataset | ADF pipeline | ADF trigger | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure SQL | Azure SQL Database | Azure Synapse | Azure Synapse Analytics | Blob Storage | CI/CD | Cost Optimization | Data Factory | Data Flows | Data Lake Storage Gen2 | Data orchestration | Dependency management | ELT | ETL | Error Handling | Integration runtime | Linked Service | Logging | Mapping Data | Mapping data flows | Monitoring | Performance Tuning | SQL | SQL Database | Scheduling | Self Hosted | Self-Hosted Integration Runtime | Synapse Analytics
Education
Related jobs
-
Data Engineer (MS) (Remote) INR 2040K-3380KCI/CD | Data Transformation | Data Validation | Date normalization | ETLMentorship opportunities | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India R7h ago
-
AI / LLM Integration Engineer (Remote) INR 1800K-2800KAPI Integration | Anthropic SDK | Async APIs | Benchmarking | Confidence scoring100 percent remote | Collaborative culture | Inclusive team culture | Professional growthMid-level Full TimeMaharashtra, Pune, India R7h ago
-
Data Operations Engineer INR 2040K-3380KApache Airflow | Data Pipelines | Data Refresh | Data Warehousing | Data pipelineEmergency incident response support | On-call rotationSenior-level Full TimeBengaluru, INDIA, India R15h ago
-
Computer Scientist ( Java Backend ) INR 3000K-4725KApache Kafka | CI/CD | Distributed Systems | Docker | Event DrivenSenior-level Full TimeBangalore, India R21h ago
-
Business Technology Analyst INR 3000K-4000KAWS | AWS Athena | AWS Glue | Airflow | Amazon RedshiftHeadspace access | Hybrid work | Mental health days | Retirement savings | Tuition reimbursementSenior-level Full TimeIndia, Hyderabad, Telangana R21h ago
-
Software Engineer INR 1500K-2362KAWS | Apache Flink | Apache Spark | Linux | PythonHybrid work | Remote work | Work-life balanceMid-level Full TimeRemote (IND), India R21h ago
-
Sr. Engineer - Gen AI with 4+ years(Remote) INR 2500K-5000KAWS | Audio Processing | Azure | Computer Vision | Distributed ComputingExtendable contract | Remote workSenior-level Full TimeBengaluru, KA, India R1d ago
-
Software Development Engineer I INR 2139K-4500KCI/CD | Code review | Data Structures | Data Structures and Algorithms | ElasticsearchProfessional development | Remote workSenior-level Full TimeBengaluru, KA, India R1d ago
-
Senior DevOps Engineer INR 2000K-4000KAWS | AWS Lambda | Agile | Amazon CodeDeploy | Amazon LinuxGreat place to work | Remote workSenior-level Full TimeBengaluru, KA, India R1d ago
-
Senior DevOps Engineer INR 2000K-4000KAWS | AWS CodeDeploy | AWS Lambda | AWS SDK | AgileGreat place to work | PradeepIT Academy training | Remote workSenior-level Full TimeBengaluru, KA, India R1d ago
-
Senior DevOps Engineer INR 2000K-4000KAWS | AWS Lambda | Agile | Amazon Linux | AnsibleRemote work opportunitySenior-level Full TimeBengaluru, KA, India R1d ago
-
Senior DevOps Engineer INR 2000K-4000KAWS | AWS CodeDeploy | AWS Lambda | AWS SDK | AgileRemote workSenior-level Full TimeBengaluru, KA, India R1d ago
-
Senior DevOps Engineer INR 2000K-4000KAWS | AWS CodeDeploy | AWS Lambda | Agile | Amazon LinuxCareer acceleration | Remote work | Training and academy coursesSenior-level Full TimeBengaluru, KA, India R1d ago
-
Pyspark & Databricks certified Developer INR 1632K-2300KAWS | Apache Spark | Azure | Azure Databricks | Data IngestionRemote workSenior-level Full TimeMumbai, MH, India R1d ago
-
Principal Engineer (Big Data Engineer) INR 2800K-4500KAmazon Web Services | Apache HBase | Apache Spark | Cassandra | Cloud platformRemote work flexibilitySenior-level Full TimeBengaluru, KA, India R1d ago
-
Oracle + PySpark Data Engineer (Remote) INR 1500K-2500KData Modeling | Data Quality | Data Warehousing | Distributed Computing | ETLDocumentation support | Remote workMid-level Full TimeBengaluru, KA, India R1d ago
-
Lead Software Engineer (Big Data and Azure) INR 2040K-3500KAgile | Apache Kafka | Apache Spark | Apache Storm | Azure BlobCareer growth | Great Place to Work certification | Remote work | Training coursesSenior-level Full TimeBengaluru, KA, India R1d ago
-
Senior-level Full TimeBengaluru, KA, India R1d ago
-
HVR Data Engineer|3-5 Years Exp|Remote INR 1500K-2000KAWS | Azure | Data Quality | ETL | HVRRemote work | Shift work supportMid-level Full TimeBengaluru, KA, India R1d ago
-
Dataops Engineer or database administration | Permanent Remote INR 1400K-2000KAWS | Amazon S3 | Ansible | Azure | CI/CDRemote workMid-level Full TimeBengaluru, KA, India R1d ago
-
Senior-level Full TimeBengaluru, KA, India R1d ago
-
Big Data & GCP- Senior Software Engineer INR 2000K-3900KAgile | BigQuery | BigQuery SQL | Cloud Storage | Cloud platformGreat place to work | Remote work flexibilitySenior-level Full TimeBengaluru, KA, India R1d ago
-
AWS Developer INR 1800K-3500KAPI Gateway | Amazon Web Services | CI/CD | Cloud Security | CodeBuildRemote workSenior-level Full TimeBengaluru, India R1d ago
-
AI Ops | API Integration | Agile | Automation Anywhere | Blue PrismWFH | Work from homeSenior-level Full TimeBengaluru, KA, India R1d ago
-
API Integration | AWS | AWS Step Functions | Amazon Bedrock | Backend DevelopmentLaptop provided | Medical insurance | Mentorship | No weekend work | Remote workSenior-level Full TimeIndia R1d ago