IN-Senior Associate_ Data Engineer Databricks_Data and Analytics_Advisory_ Bangalore
Bengaluru Millenia, India
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Adhere to compliance standards
- Build data pipelines
- Collaborate with AI/ML teams
- Curate datasets
- Design data pipelines
- Develop Databricks notebooks
- Develop spark jobs
- Document data flows
- Document operational runbooks
- Document technical designs
- Ensure Data reliability
- Ensure data governance
- Ensure data quality
- Implement access control
- Implement batch processing
- Implement incremental processing
- Ingest semi structured data
- Ingest structured data
- Maintain data pipelines
- Mentor junior engineers
- Optimize Spark for cost
- Optimize Spark for performance
- Perform data integration
- Perform data transformation
- Provide lineage
- Support production operations
- Troubleshoot data pipeline issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Agile | Amazon Web Services | Apache Spark | Auto Loader | Azure | CI/CD | Cloud Computing | Cloud platform | Data Governance | Data Transformation | Data Warehousing | Data integration | Data pipeline | Databricks | Delta Lake | Distributed Computing | ELT | ELT patterns | ETL | ETL/ELT | ETL/ELT patterns | Google Cloud | Google Cloud Platform | Lakehouse Architecture | PySpark | Python | SQL | Spark SQL | Streaming | Unity Catalog | Web Services
Education
Bachelor of Engineering | Bachelor of Science | Master of Business Administration | Master of Science
Roles
Related jobs
-
AWS | Airbyte | Airflow | Azure | CI/CDCollaboration across teams | Inclusive work environment | Learning opportunities | Professional growth | Remote workSenior-level Full TimeIndia1h ago
-
Senior Consultant INR 2700K-4000KAWS OpenSearch | Agile | Apache Kafka | At Least Once Delivery | At least onceSenior-level Full TimeBangalore, IN, 5600661h ago
-
Senior Consultant INR 2700K-4000KAWS OpenSearch | Agile | Apache Kafka | At Least Once Delivery | At least onceSenior-level Full TimeBangalore, IN, 5600661h ago
-
Senior Data Engineer INR 3000K-4000KA/B | A/B Testing | AWS | Apache Airflow | Apache HiveAccident insurance | Disability insurance | Financial wellness support | Flexible work schedule | Healthcare benefitsSenior-level Full TimeBengaluru, India2h ago
-
Engineering Lead,Microservices,Java Springboot /Sr. Associate Director, Technology Management INR 1500K-2400KApache Airflow | Apache Beam | Apache Flink | BigQuery | Cloud StorageMid-level Full TimePune, Maharashtra, India R14h ago
-
Sr. Cloud Engineer - Databricks/AI focus ( 5-9 years) Bangalore INR 2400K-3000KAKS | Bash | BigQuery | CI/CD | Cloud SecuritySenior-level Full TimeBengaluru, KA, India16h ago
-
Vice President - AI/ML Technical Developer INR 4000K-5199KAgentic Frameworks | Alerting | Automated testing | Cause analysis | Code reviewExecutive-level Full TimeBengaluru, Karnataka, India17h ago
-
ARM Microcontroller | Arc | C# | Clock | DMAEntry-level Full TimeBengaluru, KA, India17h ago
-
Data Engineer (Python ETL) INR 800K-2000KETL | Oracle Database | PL/SQL | Performance Tuning | PythonMid-level Full TimeIN-MH-Pune18h ago
-
Mid-level Full TimeIN-MH-Pune18h ago
-
API Design | AWS | Apache Spark | Azure | Big DataSenior-level Full TimeBangalore - Karnataka - India Pune …18h ago
-
Senior Analytics Engineer INR 3000K-4000KAWS | Cloud platform | Confluent Cloud | DBT | Data IngestionAgile team collaboration | Knowledge sharing culture | Mentorship opportunitiesSenior-level Full TimeBangalore - Karnataka - India Pune …18h ago
-
Lead - AI/ML Engineer (Generative AI Agent Systems) INR 3700K-5000KAI Agent Development Kit | AI tuning | Agent Development Kit | Agent Engine | Agentic WorkflowsSenior-level Contract Full TimePune, MH, India18h ago
-
DS / ML Engineer INR 1800K-3000KData Pipelines | Drift Detection | Inference Optimization | Information Retrieval | LLMMid-level Full TimeBangalore18h ago
-
Snowflake Data Engineer- Consultant INR 1500K-2000KAWS Glue | AWS Lambda | AWS S3 | Access policies | AirflowCorporate social responsibility programs | Discounts on products and services | Maternity leave | Networking opportunities | Paternity leaveMid-level Full TimeBangalore, Karnataka, India18h ago
-
Principal Engineer, Technical Architect INR 2000K-5000K.NET | API first | API-first design | Access Control | Agent OrchestrationSenior-level Full TimeRemote, India R19h ago
-
Principal Engineer, Technical Architect INR 2000K-5000K.NET | API Design | Access Control | Artificial Intelligence | Batch inferenceSenior-level Full TimeRemote, India R19h ago
-
Data Scientist - Senior INR 3600K-5000KAgile | Anomaly Detection | CI/CD | Classification | Computer VisionSenior-level Full TimePune, Maharashtra, India19h ago
-
Applied AI/ML Engineer INR 1800K-3000KAgent workflows | Agentic Systems | Auditability | Drift Detection | Error HandlingMid-level Full TimeBengaluru, Karnataka, India19h ago
-
ML Compiler Engineer, TPU Performance Optimizations INR 2500K-5000KC++ | CUDA | Compiler optimization | Computer Architecture | DebuggingSenior-level Full TimeBengaluru, Karnataka, India19h ago
-
Senior-level Full TimeIndia19h ago
-
AI Data Foundation Engineer INR 3000K-5000KApache Airflow | Apache Beam | Apache Kafka | Apache Spark | CI/CDSenior-level Full TimeIndia20h ago
-
Data Engineer –GCP BigQuery INR 3000K-4000KAdobe Experience | Adobe Experience Platform | BigQuery | Cloud Data | Cloud Data ArchitectureSenior-level Full TimeIndia20h ago
-
AI Agents | Amplitude | DBT | Databricks | LLMDirect visibility to senior leadership | Documentation and training support | Hybrid workSenior-level Full TimeBangalore, INDIA R20h ago
-
Associate Director (AI Hub - GDC) INR 1500K-3300KAWS | Apache Kafka | CI/CD | Cassandra | Cloud platformMid-level Full TimeHyderabad, Telangana, India21h ago