Data Engineer
Tasks
- Build and optimize data engineering workloads on Databricks
- Build ingestion workflows from databases APIs and event streams
- Configure AWS data services
- Configure connectors with Airbyte
- Containerize services with Docker and EKS
- Design and build data pipelines
- Develop dbt models for data transformation
- Document pipelines models and data dictionaries
- Ensure data quality, observability, and reliability
- Implement ETL ELT for batch and streaming
- Implement change data capture and incremental ingestion
- Model data in staging intermediate and mart layers
- Orchestrate workflows with Apache Airflow
- Write and optimize SQL queries
Perks/Benefits
- N/A
Skills/Tech-stack
AWS EC2 | AWS IAM | AWS Lambda | Aggregation pipelines | Airbyte | Amazon EKS | Amazon S3 | Apache Airflow | Bash | Batch Processing | CDC | CI/CD | Change Data Capture | CloudWatch | DBT | Data Capture | Data Governance | Data Observability | Data Quality | Databricks | Databricks Catalog | Delta Lake | Delta Live Tables | Delta Live) | Docker | ELT | ETL | Git | GitHub Actions | Great Expectations | Indexing | Infrastructure as Code | Kubernetes | Lakehouse | MongoDB | Mongodump | Mongorestore | PySpark | Python | Replica Sets | SQL | Sharding | Soda | Spark SQL | Streaming processing | Terraform | “as-code”
Education
N/A
Roles
Related jobs
-
Machine Learning Engineer 4 INR 2000K-3500KAWS | Alerting | Artificial Intelligence | Automation | AutoscalingSenior-level Full TimeNoida, India R11h ago
-
Full Stack AI Engineer - 3162 (Remote) INR 2000K-4500K.NET | AI Foundry | Agile | Azure AI | Azure AI FoundryFlexible working hours | Learning and development programs | Remote work | Rewards and recognition | Work-life balanceSenior-level Full TimeChennai, India R21h ago
-
Senior-level Full TimeRemote, India R1d ago
-
APIs | Automation | CMDB | CNAPP | CSPMEducation reimbursement | Health plans | Hybrid work | Parental leave | Remote workSenior-level Full TimeRemote - India R1d ago
-
JAVA Data Engineering/Associate Director, Data and Analytics INR 1000K-2000KApache Beam | Apache Flink | Apache Kafka | Apache Spark | Google DataflowContinuous professional development | Flexible working | Inclusive work environment | Opportunities to growMid-level Full TimePune, Maharashtra, India R1d ago
-
Senior Data Engineer with Azure (ADF/Fabric/Synapse), SQL, Data modeling, ETL/ELT, Python/PySpark INR 3000K-4000KAWS | Apache Airflow | Azure Data | Azure Data Factory | Azure SynapseHybrid work scheduleSenior-level Full TimeIND-Pune-Smartworks, India R1d ago
-
Senior Data Engineer INR 2500K-3500KAWS | ETL | GCP | Looker | Power BIHybrid work | Professional development | Remote workSenior-level Full TimeDelhi NCR R2d ago
-
Data Engineer (Elasticsearch + Datawarehousing) INR 1500K-2700KAPI | API Development | AWS | Apache Spark | AzureMid-level Full TimeIndia - Remote R2d ago
-
L1 Data Engineer - Remote INR 1500K-2400KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | CI/CDCollaborative team | Remote workSenior-level Full TimeIndia - Remote R2d ago
-
Sr Software Engineer INR 2400K-3500KAI Builder | Anomaly Detection | Azure Data | Azure Data Services | Azure DevOpsSenior-level Full TimeRemote Pune-Baroda, India R3d ago
-
Principal Software Engineer INR 2000K-4500K.NET | Active Directory | Agentic AI | Angular | App ServiceSenior-level Full TimeRemote Bangalore-Baroda, India R3d ago
-
Sr Operations Consultant INR 1800K-3000KAPI Integration | DAX | Data Modeling | Functions | Generative AISenior-level Full TimeRemote Pune-Baroda, India R3d ago
-
Senior AI Engineer INR 2755K-4590KAgent Orchestration | Anthropic | Azure AI | CI/CD | Enterprise SearchAsynchronous work | Continuous learning culture | Mentorship opportunities | Remote workSenior-level Full TimeRemote, Bangalore R3d ago
-
Assistant Director, Decision Systems Data INR 2000K-4000KAgile | Apache Airflow | CI/CD | Cloud Bigtable | Cloud DataflowExecutive-level Full TimeBangalore, Karnataka, India R4d ago
-
JAVA Data Engineering/Associate Director Software Engineering INR 1400K-2400KApache Beam | Apache Flink | Apache Kafka | Apache Spark | Data pipelineContinuous professional development | Flexible working | Inclusive workplace | Opportunities for growthMid-level Full TimePune, Maharashtra, India R4d ago
-
Senior AI Engineer - Remote INR 2500K-4200KAWS CloudFormation | Apache Tomcat | Best practices | CI/CD | CircleCIMentorship | Professional development | Remote workSenior-level Full TimeBengaluru, KA, India R4d ago
-
Senior AI Engineer - Remote INR 2500K-4200KAWS | AWS DocumentDB | Apache Tomcat | CI/CD | CircleCIDistributed team culture | Remote workSenior-level Full TimePune, MH, India R4d ago
-
Senior AI Engineer - Remote INR 2500K-4200KAWS Bedrock | AWS CloudFormation | Apache | Apache Tomcat | Azure AICareer growth opportunities | Mission-driven work | Remote workSenior-level Full TimeNew Delhi, DL, India R4d ago
-
Lead Data Integration Engineer INR 3200K-4000KAirflow | Airflow DAGs | Autosys | Bitbucket | CI/CDFlexible work arrangements | Health insurance | Hybrid work model | Life insurance | Paid time offSenior-level Full TimeHyderabad, India R4d ago
-
Senior-level Full TimeIND - Remote - Bangalore Based … R4d ago
-
Mid-level Full TimeNoida, India R4d ago
-
Senior Lead Quality Engineer (Gen AI) INR 2475K-4500KAPI Security | API Testing | API security testing | AWS | Access ControlSenior-level Full TimeBangalore, India R4d ago
-
Cloud Computing | Computer Vision | Data Pipelines | Data Quality | Data Quality EvaluationAccidental insurance | Hybrid work options | Life insurance | Medical insurance | Paid parental leaveSenior-level Full TimeBangalore, India (Hybrid) R4d ago
-
Senior Machine Learning Engineer - AI-Assisted Data Annotation INR 2400K-3500KConfidence scoring | Few-Shot Learning | Few-shot | Fine Tuning | Human-in-the-loopAccidental insurance | Flexible hours | Hybrid work | Life insurance | Medical insuranceSenior-level Full TimeBangalore, India (Hybrid) R4d ago
-
Benchmarking | Deep learning | Distributed Training | Efficient Deep Learning | Experiment trackingAccidental insurance | Flexible hours | Health insurance | Hybrid work | Life insuranceSenior-level Full TimeBangalore, India (Hybrid) R4d ago