Principal Engineer – Data Platforms & MLOps (Databricks)
Maharashtra, Pune, India
INR 300K-500K (estimate) Senior-level Full Time
Tasks
- Build and optimize ETL/ELT pipelines
- Collaborate with enterprise stakeholders on reference architectures
- Design end to end data architectures on Databricks
- Develop Spark Structured Streaming pipelines
- Implement Delta Live Tables workflows
- Implement Unity Catalog governance and security
- Manage ML model lifecycle with model registry
- Mentor engineers and lead design code reviews
- Operationalize ML models with MLflow
- Set up CI CD pipelines for ML deployment
- Tune Databricks clusters jobs and workflows
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Apache Spark | Azure | Azure DevOps | CI/CD | Cloud platform | Databricks | Databricks Lakehouse | Databricks SQL | Delta Lake | Delta Live Tables | Delta Live) | ELT | ETL | GitHub Actions | Google Cloud | Google Cloud Platform | MLflow | Model Registry | PySpark | Python | SQL | Scala | Spark Structured Streaming | Structured Streaming | Terraform | Unity Catalog
Education
N/A
Related jobs
-
Engineer II, Quality Engineering INR 1340K-1890KAPI Testing | AWS | Amazon Redshift | BigQuery | CI/CDContinuous learning | Family-friendly perks | Financial wellness programs | Flexible time off | Health care coverageMid-level Full TimeIN - NOIDA CANDOR TECHSPACE SECTOR …2h ago
-
Mid-level Full TimePune City, India12h ago
-
AI Native Full stack Developer - #2896 INR 1500K-1700K.Net Core | ASP.NET | AWS Lambda | Agile | Amazon EC2Cross-functional collaboration | Mentorship | Technical coachingMid-level Full TimeChennai, India12h ago
-
Data Engineer INR 400K-1200KApache Airflow | CI/CD | DBT | Data Quality | Dimensional ModelingOn-call rotationSenior-level Full TimeIN-KA-Bangalore13h ago
-
Python Robot QA INR 1524K-2541KAgile Software Development | Agile software | Azure DevOps | Backup and Recovery | C#Senior-level Full TimeIN-AP-Hyderabad13h ago
-
AI/ML | API Integration | Automation workflows | Chatbot Integration | Document processingEntry-level Full TimeNashik, MH, India13h ago
-
Group Manager - Data Engineering INR 1500K-2000KAPI | Authentication | BeautifulSoup | Data Governance | Data QualityMid-level Full TimeBangalore, KA, India13h ago
-
Machine Learning Engineer INR 2500K-3000KC plus plus | C# | Data Modeling | Data Structures | Deep learningMid-level Full TimeHyderabad, India14h ago
-
Associate Business Intelligence Analyst INR 850K-1200KAgile | Classification | Clustering | Data Analysis | Data PreprocessingMid-level Full TimeHyderabad, Telangana, India14h ago
-
Mid-level Full TimeGurgaon, Haryana, India14h ago
-
Linguistic Engineer INR 2200K-3000KBias Mitigation | Conversational AI | Hack | Knowledge graphs | Language ProcessingMid-level Full TimeHyderabad, India14h ago
-
Assistant Manager - Gen AI INR 1000K-1250KAPIs | AgentOps | Amazon SageMaker | Apache Airflow | Azure MLMid-level Full TimeBangalore, Karnataka, India14h ago
-
Assistant Manager - Gen AI INR 1000K-1250KAPIs | AWS | AWS SageMaker | Airflow | AzureAccess to AI research labs | Access to cloud innovation platforms | Certifications | Opportunity to lead POCsMid-level Full TimeBangalore, Karnataka, India15h ago
-
Assistant Manager - Gen AI INR 1000K-1250KAPI Design | AWS SageMaker | Airflow | Azure Machine Learning | CI/CDAccess to AI research labs | Cloud innovation platforms | Professional certificationsMid-level Full TimeBangalore, Karnataka, India15h ago
-
Assistant Manager - Gen AI INR 1000K-1350KAPIs | Amazon SageMaker | Apache Airflow | Apache Spark | Azure MLAccess to certifications | Access to cloud innovation platforms | Work on AI research labsMid-level Full TimeBangalore, Karnataka, India15h ago
-
Mid-level Full TimeBengaluru, Karnataka, India15h ago
-
Entry-level Full TimeIndia15h ago
-
Senior Lead Software Engineer - Java, Spark, Iceberg INR 1800K-2400KAWS Multi Az | AWS Multi Region | Amazon Web Services | Apache Iceberg | Apache SparkAgile team collaboration | Secure stable scalable deliverySenior-level Full TimeBengaluru, Karnataka, India16h ago
-
Senior Software Engineer (C, C++ and embedded Linux.) INR 1500K-2200KAgile | Bash | C# | C++ | Cloud ComputingSenior-level Full TimeHyderabad, TS, India16h ago
-
Sr. C++ Embedded Developer with Network INR 1800K-2500KBitbucket | C plus plus | CAN | CANopen | CMakeSenior-level Full Timebangalore, India16h ago
-
Senior-level Full Timebangalore, India16h ago
-
Associate Process Manager INR 1800K-3000KData Preprocessing | Elasticsearch | Embeddings | Faiss | FastAPIMid-level Full TimeMumbai, Maharashtra, India17h ago
-
API Integration | Agile | Alert Correlation | Ansible | ApacheSenior-level Full TimeBengaluru, INDIA, India17h ago
-
Senior AI/ML Engineer I INR 2500K-4000KAgentic AI | BERT | Continuous integration | Decoder Architecture | Distributed SystemsCareer growth opportunities | Certification programs | Continuous learning programs | Corporate NPS | Education reimbursement programsSenior-level Full TimeHyderabad, Telangana, India18h ago
-
Platform Engineering Professional - Data Engineer INR 2800K-4000KAWS | Ab Initio | Apache NiFi | Apache Spark | BigQueryCareer development | Inclusive workplace | Learning opportunitiesMid-level Full TimeBengaluru, IN, 56010318h ago