Software Engineer III - Data Engineer, Databricks
INR 2000K-2800K (estimate) Senior-level Full Time
Tasks
- Build data pipelines
- Conduct root cause analysis
- Create documentation and reusable frameworks
- Design data pipelines
- Develop ETL and ELT workflows
- Implement CI/CD practices
- Implement SDLC testing
- Implement data modeling
- Improve pipeline reliability
- Maintain batch data pipelines
- Monitor and alert data pipelines
- Optimize cost and scalability
- Perform incident triage
- Publish datasets
- Reconcile data
- Translate requirements into data solutions
- Tune spark jobs performance
- Validate data quality
Perks/Benefits
- N/A
Skills/Tech-stack
ACID | AWS | Access Controls | Airflow | Alerting | Azure | CI/CD | Caching | Cloud Storage | Data Governance | Data Lakes | Data Modeling | Data Pipeline Architecture | Data Quality | Data Validation | Data pipeline | Databricks | Delta Lake | ELT | ETL | Feature Engineering | File sizing | GCP | Git | Git-based workflows | Joins | Lakehouse | MLflow | Monitoring | Orchestration | Parquet | Partitioning | Pipeline architecture | PySpark | Python | SDLC | SQL | Spark SQL | Spark Structured Streaming | Structured Streaming | Time Travel | Version control
Education
N/A
Roles
Data Engineer | Engineer | Engineer III | Software Engineer | Software Engineer III
Related jobs
-
Senior Software Engineer, AI/ML, Google Photos Reminiscing INR 2500K-4460KAudio signal processing | C plus plus | Data Processing | Distributed Systems | Information RetrievalSenior-level Full TimeBengaluru, Karnataka, India2h ago
-
AI Adoption Customer Engineer INR 1800K-2800KAgent systems | Artificial Intelligence | Change Management | Cloud Architecture | Data GovernanceTravel 30 percentSenior-level Full TimeMumbai, Maharashtra, India2h ago
-
Gen AI Developer Specialist INR 2000K-3500KAPI Gateway | AWS Lambda | Agile | Amazon API | Amazon API GatewayMid-level Full TimeIndia3h ago
-
Operations Automation Associate INR 2000K-3500KAPI Integration | Agent Orchestration | CI/CD | Data Governance | Data LineageSenior-level Full TimeMumbai, Maharashtra, India3h ago
-
Senior Software Engineer INR 2000K-3465KAgent Design | Artificial Intelligence | Automation | Azure AI | Azure DevOpsSenior-level Full TimeBengaluru, KA, IN4h ago
-
Senior E2E Service Optimization Engineer (m/f/d) INR 2466K-3500K5G-NR | IMS | LTE | Python | QUICFlat hierarchies | Flexible working hours | Hybrid work | Modern hardware and digital workplace | Open company cultureSenior-level Full TimeAnkara (TUR), Chennai (IND), Communication Management4h ago
-
Senior - AWS Data Engineering INR 3000K-4000KAWS Glue | AWS Lambda | Amazon S3 | Amazon Web Services | Data ModelingSenior-level Full TimeBangalore, Karnataka, India4h ago
-
Software Engineering Professional - GEN AI INR 1500K-2700KAI orchestration | AIOps | APIs | Access Control | Agent FrameworksMid-level Full TimeBengaluru, IN, 5601035h ago
-
Mid-level Full TimeBangalore, Karnataka, India5h ago
-
Manager - Forward Deployed Engineer / AI Engineer INR 1800K-2500KAPIs | Agentic AI | Azure | Azure OpenAI | BERTMid-level Full TimeBangalore, Karnataka, India6h ago
-
Mid-level Full TimeGurgaon, HR, India7h ago
-
Senior-level Full TimeBangalore, IN, 5600668h ago
-
Mid-level Full TimeIndia11h ago
-
Senior-level Full TimeBangalore, Karnataka, IN India, 56008714h ago
-
Data Scientist III INR 2535K-4225KAWS | Advanced SQL | CI/CD | DBT | Data VisualizationFlexible work options | Work-life balance programsSenior-level Full TimeHyderabad, India14h ago
-
Senior-level Full TimeIndia - Bangalore - 5th floor, …14h ago
-
Systems Administration Specialist INR 2516K-3356KAPI Connect | Ansible | C# | Grafana | HiveMQ24x7 on-call supportSenior-level Full TimeBangalore, KA, IN14h ago
-
Data Engineer INR 1500K-2000KAsync Programming | Caching | Classification | Clustering | Data PipelinesMid-level Full TimeBengaluru, India14h ago
-
IN_Manager_Data Engineer_D&A_Advisory_Nagpur INR 1500K-2000KCI/CD | Data Modeling | Databricks | Delta Lake | DockerFlexibility programmes | Inclusive benefits | Mentorship | Support for wellbeingMid-level Full TimeNagpur, India14h ago
-
Alerting | Apache Spark | Azure | Azure Data | Azure Data FactoryFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeMumbai Shivaji Park, India14h ago
-
IN_Associate_AI Engineer _GCC_Advisory_Hyderabad INR 2000K-4000KAPI Development | Async Programming | Autogen | Automation orchestration | Best practicesFlexibility programs | Inclusive benefits | MentorshipSenior-level Full TimeHyderabad - Salarpuria, India14h ago
-
Agile | Angular | Bash | CI/CD | Continuous integrationEqual employment opportunities | Flexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeBengaluru Millenia, India14h ago
-
Agile | Angular | Bash | Cloud | Continuous integrationSenior-level Full TimeBengaluru Millenia, India14h ago
-
IN-Associate_Data engg _Digital Integration_Advisory_Gurgaon INR 1500K-2400KAgile | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeEntry-level Full TimeGurugram 10 C, India14h ago
-
Senior IT Data Engineer - R&D IT INR 3000K-4000KAWS CDK | AWS Glue | Agile | Amazon Athena | Amazon S3Senior-level Full TimeBangalore, India14h ago