IN-Senior Associate_ Data Engineer Databricks_Data and Analytics_Advisory_ Bangalore
Bengaluru Millenia, India
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Apply access control and security
- Build batch data processing
- Build incremental data processing
- Design data pipelines
- Develop Databricks notebooks
- Develop Delta Lake workflows
- Document operational runbooks
- Document technical designs
- Ensure Data Lineage
- Ensure data governance
- Ensure data quality
- Implement ETL
- Ingest semi structured datasets
- Ingest structured datasets
- Maintain production data pipelines
- Mentor junior engineers
- Optimize Spark jobs
- Transform datasets using Spark
- Troubleshoot pipeline issues
Perks/Benefits
Skills/Tech-stack
AWS | Agile | Amazon Web Services | Apache Airflow | Apache Hadoop | Apache Spark | Azure Data | Azure Data Factory | CI/CD | Cloud platform | Data Factory | Databricks | Delta Lake | Distributed Computing | ELT | ETL | Google Cloud | Google Cloud Platform | Lakehouse | PySpark | Python | SQL | Spark SQL | Streaming | Unity Catalog | Web Services
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
ML-Ops INR 3700K-5000KAsyncio | CI/CD | CProfile | Caching | DjangoLeadership development | Training sessionsSenior-level Full TimeMumbai, Maharashtra, India5h ago
-
Generative AI Engineer INR 1200K-2500KAI Services | AWS | Amazon SageMaker | Anthropic API | AzureCollaborative team environment | Leadership development | Training programsMid-level Full TimeMumbai, Maharashtra, India6h ago
-
Staff GenAI Engineer INR 2000K-5000KAWS | Agentic AI | Azure | CI/CD | ClassificationCollaborative team environment | Global consulting environment | Leadership development program | Mentorship programs | Training sessionsSenior-level Full TimeMumbai, MH, India6h ago
-
Mid-level Full TimeMumbai, Maharashtra, India6h ago
-
Senior-level Full TimeMumbai, Maharashtra, India6h ago
-
D&T Technical Solutions Manager - Marketing and Commercial INR 2000K-3814KAPI Design | API LED | API-led Architecture | Access Control | AgileMid-level Full TimePune, IN6h ago
-
Lead Data Engineer - Azure + Fabric INR 1200K-2700KAzure | Azure DevOps | CDC | CI/CD | Data GovernanceSenior-level Full TimeGurugram, Haryana, India7h ago
-
Databricks Engineer - Senior/ Lead INR 1500K-2800KACID | ADLS Gen2 | Apache Spark | Azure Data | Azure Data FactorySenior-level Full TimeHyderabad, TS, India7h ago
-
Data Engineer - Azure + Fabric INR 1400K-2700KAccess Control | Apache Spark | Azure Data | Azure DevOps | Azure SynapseSenior-level Full TimeGurugram, Haryana, India7h ago
-
Mid-level Full TimeNoida R7h ago
-
Senior-level Full TimeIndia8h ago
-
Staff Software Engineer (Python) INR 3700K-5000KAPI Development | Alerting | Amazon Web Services | Authentication | Batch ProcessingCollaborative team environment | Global projects | Leadership development | Training sessionsSenior-level Full TimeMumbai, MH, India8h ago
-
AI / ML Data Scientist I INR 1000K-1890KAWS | Apache Spark | Artificial Intelligence | Cloud platform | ConfluenceMid-level Full TimeBengaluru, India8h ago
-
Lead - Data Platform Engineering INR 2500K-4500KAWS | Azure | Cloud Data | Cloud Data Platforms | Customer 360Senior-level Full TimeChennai, India8h ago
-
GEN AI/Prompt engineer INR 1200K-2500KAI Agents | API Integration | API deployment | AWS Bedrock | Azure OpenAIMid-level Full TimeNoida, Uttar Pradesh, India8h ago
-
Software Engineer 2 INR 1500K-2500KAPI Design | AWS | Asynchronous Communication | Data Structures | Distributed SystemsMid-level Full TimeHybrid - Bangalore, India R8h ago
-
Lead Software Engineer (Full Stack) INR 506K.NET | AWS | Angular | Asynchronous programming | AzureRemote work flexibility | Supportive team culture | Work-life balanceSenior-level Full TimeBengaluru, Karnataka, India; Pune, Maharashtra, India8h ago
-
Senior-level Full TimeIndia, Karnataka, Bangalore8h ago
-
Senior-level Full TimeBangalore, India R9h ago
-
Senior Data Engineer INR 3000K-4000KAWS | AWS Managed Airflow | Airflow | Apache Spark | AzureMedical insurance | Remote-first workplace | Term Life and Personal Accident | Wellness allowanceSenior-level Full TimeIndia - Bangalore R9h ago
-
Mid-level Full TimeIndia, Karnataka, Bangalore9h ago
-
Airflow | Apache Spark | CI/CD | Contextual Bandits | Data DriftExposure to advanced ML technologies | Ownership across full ML lifecycle | Professional development | Supportive collaboration environmentSenior-level Full TimeIndia9h ago
-
Developer III INR 1400K-2500KAgile Scrum | Azure Data | Azure Data Factory | Cloud Storage | Cloud platformSenior-level Full TimeHitech City, TG, India10h ago
-
Senior Software Engineer - Data + AI Observability INR 1500K-3000KAIOps | AWS | Azure | C++ | Cloud platformSenior-level Full TimeBengaluru, India11h ago
-
Senior Data Engineer INR 3000K-4000KAmazon Web Services | Apache Spark | Cloud platform | Continuous Delivery | Data GovernanceSenior-level Full TimeBangalore, India; Chennai, India; Hyderabad, India; …11h ago