Data Engineer
Chennai, India; Chennai,India
INR 2200K-4000K (estimate) Mid-level Contract
Tasks
- Apply medallion architecture patterns
- Build dimensional models and aggregations
- Build streaming pipelines for real time analytics
- Collaborate with stakeholders on data requirements
- Design data pipelines for batch and streaming
- Develop ETL and ELT workflows
- Develop data transformation logic with Spark and SQL
- Ensure data lineage and auditability
- Implement change data capture patterns
- Implement data ingestion with Auto Loader and COPY INTO
- Implement data quality checks and automated testing
- Integrate data from multiple sources into unified assets
- Optimize pipeline performance and cost-efficiency
- Prepare data for analytics and machine learning workloads
- Set up data observability with logging monitoring and alerting
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | Alerting | Anomaly Detection | Apache Spark | Auto Loader | COPY INTO | Change Data Capture | Data Capture | Data Lineage | Data Modeling | Data Quality | Data Testing | Data profiling | Dimensional Modeling | ELT | ETL | Infrastructure as Code | Integration Testing | Logging | Medallion Architecture | Monitoring | Power BI | PySpark | Regression testing | SQL | Spark SQL | Streaming | Tableau | Unit Testing | “as-code”
Education
N/A
Roles
Related jobs
-
Senior Data Engineer INR 1500K-2500KAlerting | Apache Pinot | BigQuery | Cloud platform | Cost OptimizationSenior-level Full TimeBangalore, India12h ago
-
Senior Manager - C2 INR 3000K-5000KAI Observability | Docker | Embedding Models | Evaluation | Generative AISenior-level Full TimeGurugram, Haryana, India12h ago
-
GenAI Engineer INR 1200K-2500KAWS | Artificial neural networks | Azure | Data Structures | DatabricksMid-level Full TimeIndia15h ago
-
Bootloader | C# | CAN | Configuration Management | DebuggingFlexi benefits | Health insurance | Hybrid working model | Internal job rotation | Life insuranceEntry-level Full TimeBangalore, IN15h ago
-
Assistant Manager INR 1200K-2000KAPI Integration | CI/CD | Data Engineering | Data Governance | Data QualityMid-level Full TimeGurugram, Haryana, India16h ago
-
AI Engineer II INR 3000K-5000KAWS | Agile | Apache Hive | Apache Kafka | Apache SparkCareer growth | Holistic well-being support | Learning opportunities | Supportive leadership cultureSenior-level Full TimeBengaluru, KA, India17h ago
-
Lead Data Quality Governance and Strategy Engineer INR 2520K-3380KBI tools | Data Governance | Data Management | Data Modeling | Data QualityFlexible hybrid work model | Health and life insurance | Paid time offSenior-level Full TimeHyderabad, India R17h ago
-
Senior Data Integration Engineer I INR 3264K-4401KAWS IAM | Airflow | Amazon VPC | Apache Flink | Apache SparkSenior-level Full TimeBengaluru18h ago
-
Senior Data Engineer I INR 2500K-3380KAgile | Apache Airflow | Apache Hadoop | Apache Hive | Apache KafkaInclusive culture | Professional development time | Supportive team environment | Work-life balanceSenior-level Full TimeGurugram, HR, India18h ago
-
Analytics Engineer INR 1500K-3000KDBT | Data Governance | Data Modeling | Git | MetaplaneCareer developmentMid-level Full TimeIndia18h ago
-
Data Engineer INR 1340K-1680KAWS | Azure | ClickHouse | DBT | Data ArchitectureDonation matching | Flexible PTO | Flexible schedule | Health insurance | Lunch stipendMid-level Full TimeBengaluru, Karnataka, India19h ago
-
Senior Analyst-Data Science INR 2500K-4400KA/B | A/B Testing | B testing | Causal Inference | Data PreparationSenior-level Full TimeGurugram, HR, India20h ago
-
Data Engineer (SQL + Python/Java) INR 700K-1500KAWS EMR | Agile Scrum | Amazon Web Services | Apache Spark | Continuous DeliveryHybrid work flexibility | Remote work supportMid-level Full TimeGurgaon - Cyber Park, India1d ago
-
IN_Sr Associate_ AWS Devops With MLOPs Developer_Application Technology_Advisory_ PAN INDIA (early joiners) INR 3000K-4000KAWS | AWS Lambda | Amazon CloudFormation | Amazon EKS | Amazon S3Flexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India1d ago
-
Data Engineer - S/4 Analytics INR 2000K-2400KABAP RAP | AMDP | Access Control | Agile | Analytical ModelsFlexible working hours | Paid parental leave | Paid shift flexibility | Paid training and development programs | Remote work optionMid-level Full TimeBangalore RMZ-ECO WORLD, India1d ago
-
Vice President - Principal Data Engineer INR 3584K-5500KAPI | AWS | Agile | Anomaly Detection | AzureComprehensive healthcare | Flexible time off | Hybrid work model | Retirement plan | Support for working parentsSenior-level Full TimeM11 - Commers III, 43, 44 … R1d ago
-
EY - GDS Consulting - AIA - Gen AI - Senior INR 2000K-3300KAWS CloudFormation | Amazon Bedrock | Amazon Web Services | Apache Spark | AuthenticationCoaching | Continuous learning | Inclusive cultureSenior-level Full TimeChennai, TN, IN, 6000321d ago
-
Operations Sr. Assistant I INR 2500K-3500KAWS | Azure | Cloud Computing | Computer Vision | Data PipelinesSenior-level Full TimeIND - Ranjangaon - B200A, India1d ago
-
Artificial Intelligence Lead - Airbus India Innovation Centre INR 2500K-4800KAmazon SageMaker | Anomaly Detection | Computer Vision | Data Analysis | Language ModelsFlexible working arrangements | Inclusive work environmentSenior-level Full TimeBangalore (Airbus), India1d ago
-
Software Engineer IV INR 2000K-3000KApache Airflow | Apache Hudi | Apache Iceberg | Apache Spark | AvroMid-level Full TimeIndia - Bangalore-Navigator Bldg1d ago
-
Senior Software Engineer (Go / Java) INR 2000K-3000KAWS | Apache Airflow | Apache Hive | Apache Spark | AutoscalingContinued Career Development | Employee resource groups | Flexible WFH policy | Generous PTO | Paid volunteer timeSenior-level Full TimeIndia-Bangalore1d ago
-
Staff Software Engineer - Database INR 1500K-2500KAmazon Web Services | Apache Calcite | Apache Hive | Cloud platform | Database InternalsCareer development access | Employee resource groups | Flexible WFH | Generous PTO | Phone and internet reimbursementSenior-level Full TimeIndia-Bangalore-Remote R1d ago
-
AI Engineer, Director INR 1500K-6000KAWS Lambda | Amazon Bedrock | Amazon ECS | Amazon EKS | Amazon SageMakerExecutive-level Full TimeBengaluru, India1d ago
-
Senior-level Full TimeINDIA TECHNOLOGY CENTER, India1d ago
-
Senior-level Full TimeINDIA TECHNOLOGY CENTER, India1d ago