Data Engineer
Hyderabad, Noida/ Chennai, India
INR 1500K-2700K (estimate) Senior-level Full Time
Tasks
- Build data ingestion and transformation frameworks
- Collaborate with architects analysts BI developers and engineering teams
- Create reusable components and frameworks
- Deploy and manage ADF Databricks and Azure data components across environments
- Design batch and near real time processing solutions
- Design develop test and deploy data pipelines on Azure
- Develop data transformation logic using Python PySpark and SQL
- Implement CI CD for data engineering components with Azure DevOps
- Implement data quality checks error handling logging and monitoring
- Optimize Databricks workloads using Apache Spark and Delta Lake
- Optimize SQL queries Spark jobs and data pipelines
- Own data pipeline projects with minimal supervision
- Perform data analysis, profiling, and validation
- Prepare technical design and operational documentation
- Process structured semi structured and unstructured data
- Support production deployments and troubleshoot post production issues
- Translate business and technical requirements into data engineering solutions
- Troubleshoot pipeline failures performance issues and data discrepancies
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Event | Azure Event Hubs | Azure Key Vault | Azure Monitor | Azure SQL | Azure Synapse | Azure Synapse Analytics | CI/CD | Data Factory | Data Lake Storage Gen2 | Delta Lake | Delta Tables | Event Hubs | Git | Key Vault | Log Analytics | PySpark | Python | REST API | SQL | Synapse Analytics | Unity Catalog
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
Senior AI Engineer INR 2500K-4500KAPI Development | Asynchronous programming | Azure | CI/CD | COMTRADESenior-level Full TimeBangalore - Karnataka - India2h ago
-
Senior-level Full TimeBangalore - Karnataka - India2h ago
-
Senior AI Engineer INR 2500K-4500KAsync Programming | Azure | CI/CD | Data Ingestion | Data Ingestion PipelineDiversity-focused workplace | Equity and inclusion | Learning opportunitiesSenior-level Full TimeBangalore - Karnataka - India2h ago
-
Senior AI Engineer INR 2500K-4500KAPI Development | Asynchronous programming | Azure | CI/CD | Dependency managementSenior-level Full TimeBangalore - Karnataka - India2h ago
-
Senior-level Full TimeBangalore - Karnataka - India2h ago
-
Senior-level Full TimeBangalore - Karnataka - India2h ago
-
Data Engineer ( 2 to 5 Years) INR 1500K-2100KAWS CDK | AWS CloudFormation | AWS Glue | AWS Lambda | Amazon AthenaContinuous learning | Flexible working | Hybrid working | Inclusive cultureMid-level Full TimeBangalore - Karnataka - India2h ago
-
Backend Engineer (Cloud & AI Integration) INR 1500K-3000KAPI Integration | AWS | Agile | Asynchronous processing | DocumentationMid-level Contract Full TimeBengaluru, KA, India2h ago
-
Senior Software Engineer, AI/ML GenAI, Google Ads INR 2000K-5000KC++ | Data Processing | Data Storage | Debugging | Distributed SystemsSenior-level Full TimeBangalore, Karnataka, India4h ago
-
Backend Engineer (Cloud & AI Integration) INR 1500K-3000KAPI Integration | AWS | Agile | Asynchronous processing | DocumentationMid-level Contract Full TimeBengaluru, KA, India4h ago
-
Senior Data Engineer (Python, SQL, Snowflake, Devops, Azure/ AWS, Co-pilot, Kubernetes, Docker) INR 3200K-4131KAWS | Airflow | Automated Lineage | Azure | Azure BicepEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Mentorship | Volunteer time offSenior-level Full TimeChennai, TN, India4h ago
-
Senior Data Engineer – Snowflake Developer INR 2800K-4000KAWS | Access Control | Airflow | Azure | Azure DataSenior-level Full TimePune, Maharashtra, India5h ago
-
Senior II Data Engineer INR 3715K-4995KBI Dashboards | BICC | CI/CD | Data Augmentation | Data CatalogFlexible working options | Health benefitsSenior-level Full TimeIndia5h ago
-
Senior-level Full Timetelengana, India5h ago
-
DE&A - Data Engineer INR 1500K-2500KApache Airflow | Apache Beam | Apache Flink | Apache Kafka | Apache SparkMid-level Full TimeBangalore, Karnataka, India6h ago
-
AI/ML Engineer INR 1500K-2500KAPIs | Azure | Continuous integration | Data Pipelines | Distributed SystemsCareer advancement programs | Certification programs | Dental & vision coverage | Education reimbursement | Employee assistance programMid-level Full TimeBangalore, Karnataka, India R7h ago
-
Advanced Embedded Engr INR 1716K-3200KARINC 429 | Abstraction layer | Bitbucket | Board Support | Board Support PackageSenior-level Full TimeHyderabad, Telangana, India7h ago
-
Associate Program Manager INR 2000K-4000KAI Agents | AI Foundry | AI orchestration | APIs | AWS BedrockMid-level Full TimePune, Maharashtra, India8h ago
-
AI Data Engineer III INR 3000K-4000KAWS | Agile methodologies | Amazon Redshift | Apache Airflow | CI/CDEducation assistance program | Fitness benefits | Flexible working hours | Health checkups | Hybrid work environmentSenior-level Full TimeBengaluru, Karnataka, India R9h ago
-
Java, Spring Framework,AWS, DevOps practices and CI/CD pipelines,Docker and Kubernetes.. (6-9 Years)-BLR INR 1500K-3000KAWS | Apache Kafka | Best practices | CI/CD | CockroachDBSenior-level Full TimeIndia9h ago
-
Mid-level Full TimeIndia - Ahmedabad,Bangalore10h ago
-
API Integration | Agent Frameworks | Agentic Workflows | Automation platforms | Event DrivenAutonomy | Fully remote | Healthcare benefits | Professional development | U S business hours alignmentMid-level Full TimeIndia R10h ago
-
API Development | AWS | Agents | Angular | AzureHome office setup budget | International multicultural environment | Learning and development budget | Visa and relocation support | Work from homeSenior-level Full TimeIndia R10h ago
-
Mid-level Full TimeIndia (Remote) R10h ago
-
Lead Data Engineer INR 2500K-4500KAWS | AWS Glue | Access Control | Airflow | AzureCareer mobility | Cross portfolio opportunities | Leadership development | Learning opportunitiesSenior-level Full TimeIndia R12h ago