Databricks COE Lead
Hyderabad, Telangana, India
INR 2000K-3900K (estimate) Senior-level Full Time
Tasks
- Build reusable lakehouse frameworks accelerators
- Conduct technical reviews capability assessments skill development
- Define CoE vision roadmap governance operating model
- Define data modeling lakehouse marts feature engineering layers
- Define governance data lineage access control compliance
- Design batch and streaming data ingestion transformation
- Enable GenAI capabilities with Mosaic AI
- Enable ML and MLOps capabilities with MLflow Feature Store
- Establish Databricks CoE
- Implement scalable data pipelines with Spark Delta Lake Databricks workflows
- Implement security best practices RBAC IAM encryption secrets management
- Lead client workshops architecture reviews solution roadmaps
- Lead end to end Databricks lakehouse architecture
- Mentor engineering and architecture teams
- Optimize compute and storage cost performance
- Provide stakeholder advisory and executive communication
- Set up CI CD DevOps DataOps practices
- Standardize architecture patterns coding practices delivery methodologies
Perks/Benefits
- N/A
Skills/Tech-stack
Active Directory | Amazon Web Services | Apache Airflow | Apache Spark | Azure | Azure Active Directory | Azure Data | Azure Data Factory | Azure Event | Azure Event Hubs | Batch Processing | CI/CD | Cloud platform | DBT | Data Factory | Data Modeling | Data Pipelines | DataOps | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | DevOps | Encryption | Event Hubs | Feature Store | Google Cloud | Google Cloud Platform | IAM | Kafka | Lakehouse Architecture | Language Models | Large Language Models | MLOps | MLflow | PySpark | RBAC | SQL | Scala | Secrets management | Streaming processing | Structured Streaming | Terraform | Unity Catalog | Vector Databases | Web Services
Education
N/A
Related jobs
-
Senior-level Full TimeIN-KA-Bangalore3h ago
-
Gen AI Senior Architect INR 2000K-4500KAgent Orchestration | Artificial Intelligence | Context engineering | CrewAI | Fine TuningSenior-level Full TimeIN-AP-Hyderabad3h ago
-
Senior-level Full TimeHyderabad, Telangana, India4h ago
-
Forward Deployed Engineering Manager, GenAI, FSI Google Cloud INR 2000K-4500KAI | Agent systems | Data Governance | GenAI | Google CloudSenior-level Full TimeMumbai, Maharashtra, India5h ago
-
Amaze - Cloud Data Architect INR 1500K-2800KAzure Data | Azure Data Factory | Azure Data Lake | Azure Synapse | Azure Synapse AnalyticsSenior-level Full TimeIndia5h ago
-
Big Data Lead INR 1500K-2000KABAC | AWS | Airflow | Amazon Kinesis | AzureCross-team collaboration | MentoringSenior-level Full TimeIndia5h ago
-
Digital Next :: Manager - Gen AI - Product Manager INR 1800K-3300KArtificial Intelligence | Azure AI | Generative AI | Google AI | Language ModelsMid-level Full TimeBangalore, Karnataka, India5h ago
-
Product Manager - MLIO Knowledge Management INR 2500K-4500KAPI Development | AWS | Agile | Azure | Azure AISenior-level Full TimeMumbai, Maharashtra, India6h ago
-
Access Control | Business Continuity | Canonical Data | Canonical Data Forms | Data ArchitecturePaid leave | Paid volunteer time | Wellbeing programsSenior-level Full TimePune, MH, India7h ago
-
Senior Amaze - Data Architect INR 1500K-2000KAWS Glue | Amazon EMR | Amazon Redshift | Amazon Web Services | AzureSenior-level Full TimeIndia8h ago
-
Data Engineer INR 1500K-2000KApache Kafka | Apache Spark | BigQuery | Cloud Data | Cloud Data PlatformsMid-level Full TimeIndia9h ago
-
Manager - Sales Business Analytics INR 600K-1500KBusiness Intelligence | CART | Cluster Analysis | Data Mining | Data ProcessingAnnual health checkup | Dental benefits | Gender neutral policy | Group term life insurance | Inclusive health benefitsMid-level Full TimeGurugram, India10h ago
-
Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | BI semantic modelsCollaborative cross-functional environment | Employee well-being support | Flexible work arrangements | Healthcare benefits | MentorshipSenior-level Full TimeIndia11h ago
-
Product Manager – Data Publishing and Processing INR 1500K-3000KAWS | AWS Glue | AWS Lambda | Amazon EMR | Apache AirflowMid-level Full TimeHyderabad, Telangana, India11h ago
-
AI/ML Architect Associate INR 2000K-5000KAWS | Agent | Architectural Design | Artificial Intelligence | CI/CDHands on technical mentoring | Technical leadershipSenior-level Full TimePune, MH, India12h ago
-
Senior Manager – Data Science & Generative AI INR 2475K-4000KAWS | Agentic AI | Cloud Native | Data Pipelines | Distributed SystemsEqual employment opportunity | Health insurance | Life insurance | Personal accident insuranceSenior-level Full TimeBangalore, Karnataka, India (Hybrid) R16h ago
-
Senior Manager - SAP Analytics Cloud & Datasphere INR 2500K-3380KAnalytics Cloud | CI/CD | Cross-calculations | Data Blending | Data GovernanceSenior-level Full TimeIndia - Chennai16h ago
-
Senior-level Full TimeKandivali Plant (AD), Kandivali Plant (AD), …16h ago
-
AI Product Expert - Mahindra.AI INR 2000K-4000KAI | Data analytics | KPI Tracking | LLM Agents | Language ModelsDiversity and inclusionSenior-level Full TimeKandivali Plant (AD), Kandivali Plant (AD), …16h ago
-
IN_Manager_Data Engineer_D&A_Advisory_Nagpur INR 1500K-2000KCI/CD | Data Modeling | Databricks | Delta Lake | DockerFlexibility programmes | Inclusive benefits | Mentorship | Support for wellbeingMid-level Full TimeNagpur, India16h ago
-
Sr. Principal, Quality Engineering Architect INR 2000K-4000KAWS | Azure | BDD | Behavior-Driven Development | Bias detectionSenior-level Full TimeField - India16h ago
-
IN_Manager_Data Engineer_D&A_Advisory_Nagpur INR 1500K-2000KAzure | CI/CD | Databricks | Delta Lake | DockerMid-level Full TimeNagpur, India16h ago
-
Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure DevOpsFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeMumbai Shivaji Park, India16h ago
-
Manager - Data Analyst INR 2000K-2755KDashboards | Data Analysis | Data Management | Data Visualization | Data cleaningFlexible benefits | Travel up to 25 percentSenior-level Full TimeGH Office: Chennai, India (TRIL)16h ago
-
Lead Product AI Data Engineer & Architect INR 2000K-3300KAWS | Apache Airflow | Azure | Cloud platform | DBTEqual employment opportunity | Full-time employment | Hybrid working environment | Professional training opportunitiesSenior-level Full TimeR131-Bangalore, India16h ago