Data Engineer
Tasks
- Apply data fabric design principles including metadata driven pipelines orchestration lineage governance
- Collaborate with cross functional teams on data architecture and best practices
- Design and implement data ingestion pipelines with Azure Data Factory
- Implement RAW and curated data layer workflows
- Monitor collect and analyze pipeline performance metrics
- Translate T SQL transformation logic to Databricks PySpark
- Tune and optimize Databricks read write operations partitioning file formats query optimization
Perks/Benefits
Skills/Tech-stack
Azure Data | Azure Data Factory | Data Factory | Data Governance | Data Lineage | Data fabric | Databricks | File Formats | Metadata Driven | Metadata Driven Pipeline | Orchestration | Partitioning | Performance Tuning | PySpark | Query Optimization | T-SQL
Education
N/A
Roles
Related jobs
-
Consultant_ GCP Data Engineer INR 1200K-2000KAirflow | BigQuery | Cloud Composer | Cloud Functions | Cloud StorageAI certifications | Training and mentorshipEntry-level Full TimeIndia-Gurugram9h ago
-
AI/ML Engineer INR 2500K-4000KAzure Cognitive | Azure Cognitive Search | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeIN-HR-Gurgaon9h ago
-
AIML Engineer INR 2500K-4000KAzure Data | Azure Data Factory | Azure Data Lake | Cognitive Search | Data FactorySenior-level Full TimeIN-HR-Gurgaon9h ago
-
Pyspark Databricks AWS INR 500K-2000KAWS Glue | Agile | Airflow | Amazon Web Services | Apache KafkaSenior-level Full TimeIN-TN-Chennai9h ago
-
Senior-level Full TimeIN-TN-Chennai9h ago
-
AI/ML-Data Scientist INR 2500K-3500KAzure Data | Azure Data Factory | Azure Data Lake | Azure OpenAI | Cognitive SearchSenior-level Full TimeIN-HR-Gurgaon9h ago
-
Mid-level Full TimePune, Maharashtra, India11h ago
-
AWS Lead Data Engineer INR 2000K-2040KAWS | AWS Glue | AWS architecture | Agile methodology | Amazon AuroraSenior-level Full TimeGurugram, Haryana, India11h ago
-
DE&A - Core - Cloud Data Engineering - Azure Data Engineering INR 1500K-2000KAggregation pipelines | Azure Blob | Azure Blob Storage | Azure Cosmos | Azure Cosmos DBMid-level Full TimePune, Maharashtra, India12h ago
-
Lead Software Engineer - AWS, PySpark, BigData INR 2000K-2450KAWS | AWS CDK | AWS Lambda | Agile | AirflowSenior-level Full TimeHyderabad, Telangana, India12h ago
-
Software Engineer III - AWS, PySpark, ETL INR 2000K-2450KAWS EMR | Agile | Apache Spark | CI/CD | Data ModelingSenior-level Full TimeHyderabad, Telangana, India12h ago
-
Software Engineer III - PySpark, BigData, AWS INR 2050K-2600KAWS | Agile | Amazon Web Services | Application Resiliency | Application SecuritySenior-level Full TimeHyderabad, Telangana, India13h ago
-
Azure Data Engineer - Associate Consultant INR 1500K-2000KAzure Data | Azure Data Factory | Azure Data Flow | Azure Data Lake | Azure DatabricksMid-level Full TimeBangalore, Karnataka, India14h ago
-
Data Engineer INR 960K-1400KADO | Automated testing | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeMumbai, Maharashtra, India16h ago
-
Sr.Manager Software Engineer INR 2050K-3000KAWS EKS | Agent Development | Amazon ECS | Amazon EMR | Amazon Web ServicesSenior-level Full TimeBengaluru, Karnataka, India17h ago
-
Applied AI ML Sr Associate INR 2500K-3500KAWS Lambda | AWS S3 | AWS Step Functions | Agentic AI | Amazon AthenaSenior-level Full TimeHyderabad, Telangana, India17h ago
-
Machine Learning Data Engineer III INR 2500K-5000KAPI Development | AWS | Amazon Bedrock | Application development | DatabricksSenior-level Full TimeHyderabad, Telangana, India17h ago
-
Applied AI ML Associate INR 1068K-1496KAWS Lambda | AWS S3 | AWS Step Functions | Agentic AI | Amazon AthenaMid-level Full TimeHyderabad, Telangana, India17h ago
-
Data Governance Engineer INR 2500K-3100KAPI Integration | Business glossary | CRM | Collibra | Data CatalogSenior-level Full TimeIND-Tamil Nadu-Chennai-KLA, India22h ago
-
AI/Data Science Engineer II INR 1100K-1890KAWS | AWS Glue | AWS S3 | Agentic Systems | AlertingFlexible benefits package | Short term incentive called Medtronic Incentive PlanMid-level Full TimeIND-TS Hyderabad Nanakramguda, India22h ago
-
Senior-level Full TimeHyderabad TS IN 26, India22h ago
-
Lead Developer (SQL,Python) INR 2500K-3380KAWS | Cost Optimization | DBT | Data Modeling | Data QualitySenior-level Full TimeVirtual Office (Telangana), India R22h ago
-
Data Ops Engineer INR 1431K-2040KApache Airflow | CI/CD | DBT | Dagster | Data GovernanceAccident insurance | Employee assistance program | Employee recognition awards | Group term life insurance | Health insuranceMid-level Full TimeBangalore, KA, IN, 560 02922h ago
-
Data Engineer INR 1500K-2000KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCareer growthMid-level Full TimeBengaluru, India22h ago
-
Palantir Lead INR 1500K-2400KAPI Integration | AWS | Access Control | Apache Spark | Batch Data ProcessingSenior-level Full TimePune, India22h ago