Data Engineer
Tasks
- Build data pipelines
- Design data pipelines
- Enforce data quality
- Enforce data security
- Ensure data compliance
- Implement batch processing
- Implement data governance
- Implement streaming processing
- Ingest data
- Integrate with Azure Data Lake Storage
- Integrate with Azure services
- Maintain data pipelines
- Monitor pipeline performance
- Process large-scale datasets
- Report compute cost anomalies
- Support databricks users
- Track compute costs
- Transform data
- Troubleshoot pipeline issues
- Tune Databricks jobs
- Tune Spark configurations
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Kafka | Apache Spark | Azure | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Machine Learning | Data Compliance | Data Governance | Data Lake Storage | Data Security | Databricks Runtime | Databricks SQL | Databricks Workflows | Delta Lake | ELT | ETL | Gold Medallion | MLflow | Machine Learning | MuleSoft | PySpark | Python | SQL | Scala
Education
Roles
Related jobs
-
Staff Engineer, Data Engineering INR 2400K-3100KAPIs | Airflow | Apache Kafka | Apache Spark | AzureSenior-level Full TimeBengaluru, KA, India3h ago
-
Mid-level Full TimeHyderabad, TS, IN; Bengaluru, KA, IN3h ago
-
Sr. Staff Engineer (Java Fullstack or Python, AI) INR 2475K-3500KAWS | Angular | Anthropic | Azure OpenAI | Bias MitigationFamily well-being benefits | Health benefits | Paid time off | Work-life balanceSenior-level Full TimeHyderabad, India3h ago
-
Data Engineer INR 1000K-2100KApache Airflow | Apache Spark | Azure | CI/CD | DatabricksEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India5h ago
-
Senior Software Engineer (Microsoft .NET/Azure cloud) INR 2829K-4000K.NET | API Management | APIM | Angular | App ServicesHealth insurance | Paid time off | Wellbeing benefits | Work-life balanceSenior-level Full TimeHyderabad, India5h ago
-
Consultant - AI Engineer INR 2000K-3500KAutogen | CI/CD | Claude | Cloud infrastructure | ContainerizationSenior-level Full TimeBangalore, Karnataka, India6h ago
-
MLOps Engineer (Generative AI) INR 1500K-2100KAgentic pipelines | Agile | Amazon Web Services | Azure | Azure Machine LearningEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India6h ago
-
Machine Learning (Generative AI) INR 1500K-2500KAgile | Azure Machine Learning | CI/CD | Computer Vision | ConfluenceEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India6h ago
-
Generative AI Engineer INR 2000K-3500KAI Studio | AWS SageMaker | Agents | Amazon Bedrock | Anthropic APICollaborative team environment | Leadership development | Training programsMid-level Full TimeMumbai, Maharashtra, India6h ago
-
Staff GenAI Engineer INR 2000K-3500KAWS | Agentic AI | Azure | CI/CD | Distributed SystemsCollaborative team environment | Leadership development programs | Training sessionsSenior-level Full TimeMumbai, MH, India6h ago
-
API Testing | Apache Airflow | Azure | CI/CD | DatabricksEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, India7h ago
-
Tech Manager- GTM Applied AI & Analytics INR 2000K-3380KAirflow | Apache Spark | Databricks | Fine Tuning | LangchainFlexible work schedule | Hybrid work scheduleSenior-level Full TimeBengaluru, KA, India7h ago
-
Mid-level Full TimeGurugram, India7h ago
-
Software Test Engineer- Must have- ETL Testing, Pyspark, SQL, Good to have-playwright or selenium, databricks or snowflake INR 1632K-2156KAzure | Bitbucket | CI/CD | Databricks | DockerEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimePune, MH, India7h ago
-
AI/ML Expert INR 2500K-4000KARIMA | Anomaly Detection | Computer Vision | Convolutional Neural Network | Data AnalysisSenior-level Full Timecoimbatore, India8h ago
-
AM - Cloud Data Engineer INR 1500K-2400KAWS Lake Formation | Apache Airflow | Apache Kafka | Apache Spark | Azure CosmosClient-facing consulting experience | Growth mindset | Professional development opportunitiesSenior-level Full TimePune, Maharashtra, India8h ago
-
Data Scientist I INR 302K-420KAnomaly Detection | Cloud Computing | Data cleaning | Feature Engineering | Generative AIEntry-level Full TimeBangalore, KA, IN, 5600768h ago
-
Data Operation Engineer INR 2040K-3000KAsset bundles | Azure | Cloud platform | Confluence | DBTHybrid work arrangementMid-level Full TimeChennai, Tamil Nadu, India9h ago
-
AWS | Azure | Cloud platform | Deep learning | Generative AIContinuous learning and growth | Fully remote work | Technical excellence and experimentation cultureSenior-level Full TimeIndia R9h ago
-
Senior Data Engineer, Payments INR 3050K-4350KAWS Glue | Amazon Redshift | Apache Airflow | Apache Flink | Apache KafkaSenior-level Full TimeBangalore, India9h ago
-
AWS | Access Control | Apache Airflow | Azure | CI/CDAutonomy | Career growth | Inclusive culture | Learning and developmentSenior-level Full TimeIndia9h ago
-
Senior - AWS Data Engineering INR 3000K-4000KAWS | Data Integrity | Data Modeling | Data Validation | Data throughputSenior-level Full TimeBangalore, Karnataka, India9h ago
-
AI/ML Expert INR 2500K-4800KARIMA | Anomaly Detection | Computer Vision | Convolutional Neural Networks | Data AnalysisSenior-level Full Timecoimbatore, India9h ago
-
AI/ML Expert INR 2500K-4800KARIMA | Anomaly Detection | Computer Vision | Convolutional Neural Networks | Data AnalysisSenior-level Full Timecoimbatore, India9h ago
-
AI/ML Expert INR 2500K-4800KARIMA | Anomaly Detection | Computer Vision | Convolutional Neural Networks | Data AnalysisSenior-level Full Timecoimbatore, India9h ago