Expert Data Engineer (Databricks)
Maharashtra, Pune, India
INR 1500K-2400K (estimate) Senior-level Full Time
Tasks
- Build BI dashboards
- Build ETL and ELT pipelines
- Build inference pipelines
- Design data pipelines
- Design schemas and tables
- Implement CI/CD pipelines
- Implement data governance and lineage
- Implement data validation and monitoring
- Implement model versioning
- Manage enterprise data catalog
- Manage model deployment lifecycle
- Optimize compute and storage cost
- Orchestrate workflows
- Productionize machine learning models
- Set up alerting and automated recovery
- Use infrastructure as code
Perks/Benefits
- N/A
Skills/Tech-stack
API Design | API Key | AWS Glue | AWS IAM | AWS Lambda | AWS S3 | AWS VPC | Alerting | Apache Spark | Authentication | Authorization | CI/CD | Data Lakehouse | Databricks | Databricks Workflows | Delta Lake | FastAPI | Flask | Infrastructure as Code | MLOps | Machine Learning | Monitoring | OAuth | PySpark | Python | Query Optimization | REST API | Unity Catalog | “as-code”
Education
N/A
Related jobs
-
APIs | AWS | Algorithms | Azure | CI/CDSenior-level Full TimeKolkata DN 57, India4h ago
-
Senior-level Full TimeHyderabad, IND-TG, India11h ago
-
Senior-level Full TimeHyderabad, IND-TG, India11h ago
-
Data Engineer INR 1500K-2500KAgile | BeautifulSoup | Browse AI | CSV | Data PipelinesHybrid work | Paid time offMid-level Full TimeHyderabad, India12h ago
-
Senior Automation Specialist INR 1500K-3000KAPI Integration | Access Controls | Alerting | Data Quality | MonitoringSenior-level Full TimeHyderabad, India14h ago
-
Software Engineer (ETL TESTER - DATA WAREHOUSING, DATA PROCESSING, SQL, UAT, SNOWFLAKE) INR 1000K-1500KAirflow | Azure Data | Azure Data Factory | Batch Processing | Control-MEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offMid-level Full TimeChennai, TN, India14h ago
-
AI QA Engineer INR 2000K-3300KAWS Bedrock | Azure OpenAI | Case design | Content Filtering | Evaluation rubricsImmediate joiner required | Work From Office 2 DaysSenior-level Full TimePune City, India15h ago
-
Mid-level Full TimeIN-AP-Hyderabad15h ago
-
Data Engineering Manager INR 1500K-2500KADLS | Access Management | Azure | Azure Data | Azure Data FactoryMid-level Full TimeBangalore, Karnataka, India16h ago
-
GCP Data Engineer - Associate Director INR 800K-2000KApache Airflow | Apache Kafka | Apache Spark | BigQuery | CI/CDMid-level Full TimeHyderabad, Telangana, India16h ago
-
AI Engineer INR 1500K-3000KAzure Databricks | Data Engineering | Data engineering pipelines | Embeddings | Engineering PipelinesMid-level Full TimeBengaluru, Karnataka, India17h ago
-
Senior Engineer Digital & AI Solutions INR 4000K-6000KAPI Development | Agent systems | CI/CD | Cloud Computing | ContainerizationFood allowance | Health insurance | Hybrid work | Internet allowance | Learning and development programsSenior-level Full TimeHyderabad, Telengana, India17h ago
-
Manager of Software Engineering-Java, Flink,Kafka INR 1500K-2200KAWS | Agile | Amazon EKS | Apache Flink | Apache KafkaMid-level Full TimeHyderabad, Telangana, India17h ago
-
Software Engineer II,Python, Spark INR 1500K-2200KAWS | AWS Glue | AWS Glue Catalog | AWS KMS | AWS Lake FormationMid-level Full TimeMumbai, Maharashtra, India18h ago
-
AI QA Engineer - Bangalore, IND INR 2000K-3000KAPI Development | Claude | Docker | Embeddings | Evaluation FrameworksMid-level Full TimeIndia18h ago
-
Lead Software Engineer - Java/Python, AWS,LLM INR 3500K-5000KA/B | A/B Testing | AWS Lambda | Agentic AI | AgileSenior-level Full TimeMumbai, Maharashtra, India18h ago
-
Staff Engineer, Data Analytics Engineering INR 2500K-3500KAPIs | AWS Glue | AWS Lambda | AWS S3 | Amazon BedrockSenior-level Full TimeBengaluru, KA, India18h ago
-
Staff Data Scientist INR 2000K-4500KAlgorithms | Data Lakes | Data Structures | Deep learning | DockerSenior-level Full TimeBangalore, India Office (BANGALORE) R19h ago
-
Senior Associate - Data Science/Applied AI ML INR 1400K-2400KAnomaly Detection | Backtesting | Bias/fairness | CI/CD | ClassificationMid-level Full TimeMumbai, Maharashtra, India21h ago
-
Senior Associate - Data Science/Applied AI ML INR 1400K-2400KAnomaly Detection | Bias and fairness | CI/CD | Classification | ClusteringMid-level Full TimeMumbai, Maharashtra, India21h ago
-
Data Engineer (GCP) INR 1500K-2040KAPIs | Airflow | Authentication | Authorization | BigQueryAgile collaboration | Autonomy | Flexible environmentMid-level Full TimePune, Maharashtra, India21h ago
-
Senior-level Full Timetelengana, India21h ago
-
Associate Specialist - Cloud & Application Engineering INR 800K-1000KAWS | Azure | Cloud Data | Cloud Data Platforms | Cloud platformMid-level Full TimeMumbai, Maharashtra, India21h ago
-
Associate Principal Engineer, Data Engineer (Enterprise Data Architecture- Banking domain) INR 1500K-2500KArdoq | Cloud Data | Cloud Data Platforms | Collibra | Data ArchitectureMid-level Full TimeRemote, India R22h ago
-
Consultant - Python Developer with Gen AI INR 1500K-2500KAI Search | Agile | Azure OpenAI | CSS | DjangoMid-level Full TimeBangalore, Karnataka, India22h ago