Data Engineer - VOIS
Tasks
- Configure error handling parameterization and triggers in Cloud Data Fusion pipelines
- Design and build data pipelines on GCP using Databricks
- Develop reusable Python pipeline components
- Implement data quality and observability using validation and monitoring
- Maintain monitoring and performance tuning for data products
- Optimize pipeline performance reliability and cost
- Orchestrate ETL ELT and feature engineering workflows
Perks/Benefits
Skills/Tech-stack
Apache Airflow | Apache Spark | BigQuery | CI/CD | Cloud Composer | Cloud Data | Cloud Data Fusion | Cloud Storage | Data Fusion | Databricks | Dataproc | Delta Lake | GitLab CI | Google BigQuery | Linting | PubSub | PySpark | Python | SQL | Type hinting | Unit Testing | Unity Catalog
Related jobs
-
Sr. Software Engineer (AI/ML) (PhD / Postdoc) INR 1500K-2400KCausal Inference | Deep learning | Generative AI | Language Processing | Machine LearningAccess to large clinical dataset | Collaboration with healthcare partners | Conference publication support | Research freedomSenior-level Full TimeHyderabad, India5h ago
-
Business Intelligence Engineer, Last Mile Analytics INR 1800K-2400KAmazon QuickSight | Data Analysis | Data Interpretation | Data Visualization | NoSQLMid-level Full TimeHyderabad, Telangana, IND11h ago
-
IT Data Engineer II INT INR 1800K-2500KApache Spark | Cloud Cost Optimization | Cloud platform | Cost Optimization | Data GovernanceMid-level Full TimeBangalore, India11h ago
-
AWS CloudFormation | Amazon Kinesis | Amazon Redshift | Amazon S3 | Amazon Web ServicesEqual opportunity employer | Fast track growth opportunities | Great place to workSenior-level Full TimeHyderabad, Office Level 3 & 4, …11h ago
-
TTT-SAP-Python Gen AI-Tax Senior INR 2000K-4512KAI Core | AI Launchpad | API Management | CI/CD | Cloud ALMSenior-level Full TimeBengaluru, KA, IN, 56001611h ago
-
Entry-level Full TimeBangalore, India11h ago
-
Senior-level Full TimeBangalore, KA, IN11h ago
-
Deputy Manager, Global Platform Technology INR 1500K-2000KAgile | Azure Blob | Azure Blob Storage | Azure Cosmos | Azure Cosmos DBMid-level Full TimeChennai IMPT, India11h ago
-
Sr. Big Data Engineer INR 2400K-3500KAWS | Apache Spark | Azure | Cloud Platforms | Data ArchitectureTravel opportunitiesSenior-level Full TimeRemote - India R19h ago
-
Machine Learning Engineer - 2 INR 2000K-2500KAWS | Algorithms | Computer Vision | Data Pipelines | Data StructuresEquity | Family insurance | Flexible hours | Free health teleconsultations | Hybrid workMid-level Full TimeBangalore, India1d ago
-
Data Infrastructure Engineer INR 2000K-4000KAWS | Apache Flink | Apache Kafka | Apache Spark | Batch ProcessingEquipment | Health insurance | Meals | Meaningful equity | Onsite collaboration cultureSenior-level Full TimeBengaluru1d ago
-
Senior-level Full TimeNoida1d ago
-
Senior Machine Learning Engineer - Research INR 2500K-4500KAWS | Azure | BERT | Cloud platform | ClusteringSenior-level Full TimeNoida1d ago
-
Mid-level Full TimeBangalore, Karnātaka, India1d ago
-
Data Preprocessing | Data Visualization | Econometrics | Machine Learning | Microsoft ExcelMid-level Full TimeIndia R1d ago
-
Senior Data Engineer INR 3000K-4000KAPIs | AWS | AWS Glue | AWS Lambda | AgileAccess to Headspace app | Career development | Hybrid work model | Incentive programs | Mental health daysSenior-level Full TimeIndia, Bengaluru, Karnataka R1d ago
-
Manager, Data Platform INR 1500K-2700KAPI | API Gateway | AWS | CI/CD | CloudWatchFlexible vacation | Flexible work schedule | Headspace access | Hybrid work model | Mental health daysMid-level Full TimeIndia, Bengaluru, Karnataka R1d ago
-
Software Engineer, Data Platform INR 1500K-3000KAPI | API Gateway | AWS | CI/CD | CloudWatchEmployee incentive programs | Flexible vacation | Headspace app | Hybrid work model | Mental health daysMid-level Full TimeIndia, Bengaluru, Karnataka R1d ago
-
EY - GDS Consulting - AI and DATA - AWS Databricks - Senior INR 3000K-4000KAWS | AWS Glue | AWS Lambda | AWS Step Functions | Amazon S3Coaching and learning programs | Flexible work arrangements | Global knowledge exchange | Inclusive & Supportive Culture | Personalized growth planSenior-level Full TimeBengaluru, KA, IN, 5600161d ago
-
Software Developer / Software Engineer INR 960K-1935KAutomation | DBMS | Database | Excel | File handlingMid-level Full TimeTN, Chennai, India1d ago
-
Senior Staff Engineer, Gen AI, Azure INR 2000K-4000KAI/ML | API Development | Azure AI | Azure AI/ML | Backend DevelopmentSenior-level Full TimeRemote, India R2d ago
-
Associate Principal Engineer, Azure Data Factory Architect INR 1800K-2500KADF Pipelines | Azure Blob | Azure Blob Storage | Azure Data | Azure Data FactoryMid-level Full TimeRemote, India R2d ago
-
Senior Staff Engineer,GenAI, Playwright INR 1800K-2829KAI Agents | API Testing | Agile | BDD | CI/CDSenior-level Full TimeRemote, India R2d ago
-
Sr Technical Lead-App Development INR 2500K-4500KAg Grid | Amazon Web Services | Autogen | Azure | Cloud platformSenior-level Full TimeINDIA - PUNE - BIRLASOFT OFFICE …2d ago
-
AI Vision Engineer INR 360K-420K3D Vision | Active Learning | Annotation | Camera systems | Cloud processingFamily-friendly benefits | Inclusive environmentEntry-level Full TimeIND-Tamil Nadu-Chennai-KLA, India2d ago