Databricks Engineer - Senior/ Lead
Tasks
- Apply Azure security and RBAC governance
- Build CI/CD deployment pipelines
- Build SQL transformations for analytics and ML
- Design scalable data pipelines
- Develop ETL/ELT workflows
- Develop streaming data solutions
- Implement data quality validation monitoring and error handling
- Improve pipeline architecture engineering standards and operational efficiency
- Integrate Databricks with Azure Data Lake Storage Gen2
- Maintain Delta Lake tables
- Optimize Spark jobs for performance and scalability
- Process structured and semi-structured data
- Provide technical guidance on data engineering best practices
- Support data modeling for analytics
- Use Git/version control
Perks/Benefits
- N/A
Skills/Tech-stack
ACID transactions | Apache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure Databricks | Azure DevOps | Azure Event | Azure Event Hubs | Azure Security | CI/CD | CSV | Data Governance | Data Lake Storage Gen2 | Delta Lake | Delta Lake ACID | Delta Lake ACID transactions | Delta Lake Schema | Delta Lake Time Travel | Delta Lake schema evolution | ELT | ETL | Event Hubs | Git | JSON | Kafka | Parquet | PySpark | Python | RBAC | SQL | Schema evolution | Spark SQL | Spark Structured Streaming | Structured Streaming | Time Travel
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Senior Data Engineer
Related jobs
-
Senior Software Engineer, Machine Learning INR 3600K-5400KA/B | A/B Testing | Amazon S3 | Apache Airflow | B testingDental insurance | Financial wellness support | Health insurance | Mental health support | Paid time offSenior-level Full TimeBengaluru, India R2h ago
-
Senior Staff Engineer INR 2500K-4000KAlgorithms | Amazon Web Services | Big Data | Cloud Computing | Code reviewCustomer-focused culture | Flexible work arrangementsSenior-level Full TimeBangalore3h ago
-
Senior Software Engineer(AI/ML), Trust INR 4500K-6500KA/B | A/B Testing | Anomaly Detection | Apache Airflow | Apache KafkaSenior-level Full TimeBangalore, India3h ago
-
Apache Airflow | Apache Spark | Batch Processing | Cloud infrastructure | Data GovernanceHigh impact ownership | Professional growth opportunities | Remote first working environment | Technical leadership opportunitiesSenior-level Contract Full TimeIndia4h ago
-
ABAP | BDC | Business Intelligence | Data Modeling | Data PipelinesCareer development | Employee communities | Global onboarding | Health insurance | Hybrid workSenior-level Full TimeIndia R4h ago
-
API Versioning | Asynchronous processing | Authentication | CI/CD | CeleryCollaborate with backend and frontend teams | Full-time employment | Fully remote | Hands on experience with production LLM and VLM systems | Work on AI infrastructure model routing and microservicesMid-level Full TimeIndia R5h ago
-
CI/CD | Deep learning | Edge Computing | Experiment tracking | Hugging FaceSenior-level Full TimeIndia7h ago
-
Associate Director, Machine Learning Engineering INR 1000K-1890KAmazon Web Services | Computer Vision | Data Preprocessing | Deep learning | Feature EngineeringContinuing education program | Continuous learning | Family-friendly perks | Financial wellness programs | Flexible time offMid-level Full TimeIN - GURUGRAM DLF DOWNTOWN BLOCK …12h ago
-
Agentic AI Engineer (ITSM & Enterprise Automation) INR 2515K-4680KAPI Authentication | Artificial Intelligence | Authentication | Authorization | Change ManagementCareer growth and developmentSenior-level Full TimePune13h ago
-
Agentic AI Engineer (ITSM & Enterprise Automation) INR 2500K-4680KAPI Authentication | API authorization | Asynchronous Workflows | Authentication | AuthorizationAnnual bonus opportunity | Career growth and development | Flexible work arrangementSenior-level Full TimePune13h ago
-
Machine Learning Engineer IV INR 4443K-7998KAPIs | Batch Processing | Cost Optimization | Distributed Systems | EmbeddingsBonuses | Disability insurance | Health insurance | Life insurance | Paid parental leaveSenior-level Full TimePune, India R13h ago
-
ML Developer (with Computer Vision) INR 3000K-5000KAWS | Algorithms | Azure | Computer Vision | Convolutional Neural NetworksMid-level Full TimePune City, India16h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAgentic AI | Amazon Web Services | Azure | CI/CD | Cloud SecurityAnnual learning budget | Flexible hours | Growth opportunities | Internal TechTalks | MentorshipSenior-level Full TimeKolkata, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAPI Integration | AWS | Anthropic | Azure | CI/CDAnnual learning budget | Flexible hours | Internal technical talks | Mentorship | People focused supportSenior-level Full TimeNoida, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | AWS S3 | Azure | CI/CD | Cloud SecurityCollaborative culture | Flexible hours | Internal TechTalks | Learning budget | MentorshipSenior-level Full TimeTrivandrum, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | Agentic AI | Azure | CI/CD | Cloud SecurityCollaborative culture | Flexible hours | Internal technical talks | Learning budget | MentorshipSenior-level Full TimeHyderabad, India17h ago
-
Lead Agentic AI Engineer ID80141 JPY 8600K-10000KAWS | Agentic AI | Azure | CI/CD | Cloud SecurityAnnual learning budget | Flexible hours | Internal tech talks | Mentorship | Performance and compensation reviewsSenior-level Full TimeKochi, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | Agentic AI | Azure | CI/CD | Cloud infrastructureFlexible hours | Internal technical talks | Learning budget | Mentorship | People focused supportSenior-level Full TimeNew Delhi, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | Azure | CI/CD | Cloud Security | Cloud infrastructureCollaborative culture | Flexible hours | Internal tech talks | Learning budget | MentorshipSenior-level Full TimeMumbai, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | Agentic AI | Azure | CI/CD | Cloud SecurityChallenging projects | Collaborative culture | Flexible hours | Internal TechTalks | Learning budgetSenior-level Full TimeBangalore North, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | Agentic AI | Azure | CI/CD | Cloud SecurityAnnual learning budget | Collaborative culture | Flexible hours | Internal TechTalks | MentorshipSenior-level Full TimeCoimbatore South, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | Agentic Models | Azure | CI/CD | Cloud SecurityAutonomy | Challenging projects | Collaborative culture | Flexible hours | Internal TechTalksSenior-level Full TimeChennai, India17h ago
-
Lead Agentic AI Engineer ID80141 INR 2500K-5000KAWS | Agentic Models | Azure | CI/CD | Cloud SecurityAnnual performance and compensation reviews | Flexible hours | Internal TechTalks | Learning budget | MentorshipSenior-level Full TimeIndore, India17h ago
-
Senior Agentic AI Engineer ID80140 INR 2000K-5000KAmazon Web Services | Anthropic | Azure | CI/CD | Cloud SecurityAnnual learning budget | Flexible hours | Mentorship | Remote work | Well-being programsSenior-level Full TimeKolkata, India17h ago
-
Senior Agentic AI Engineer ID80140 INR 2000K-5000KAnthropic API | CI/CD | Cloud Security | Drift Detection | Infrastructure as CodeAnnual learning budget | Collaborative culture | Flexible hours | Mentorship | Performance reviewsSenior-level Full TimeAhmedabad, India17h ago