Sr. Data Architect (Databricks)
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Architect AI and ML platforms using Databricks
- Architect batch and real time streaming data pipelines
- Automate deployment of notebooks and jobs
- Build semantic models for reporting and analytics
- Configure row level and column level security
- Define CI/CD strategies
- Define enterprise data architecture
- Define technical standards and best practices
- Design ETL and ELT frameworks using Apache Spark and Databricks
- Design Medallion Architecture
- Design and optimize Delta Lake
- Design and optimize Delta Live Tables
- Design data pipelines for machine learning workloads
- Design end to end Lakehouse architectures using Databricks
- Design enterprise data models
- Develop scalable secure and highly available data platforms
- Enable ML lifecycle management with MLflow
- Establish metadata management lineage and auditing
- Establish version control and branching strategies
- Implement Unity Catalog for centralized governance
- Implement change data capture and incremental processing
- Implement infrastructure-as-code
- Implement performance and cost optimization
- Lead architecture reviews and design workshops
- Lead cloud data modernization and migration
- Mentor data engineers and technical leads
- Optimize Spark jobs and SQL workloads
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure Key Vault | Azure Synapse | Azure Synapse Analytics | CI/CD | Change Data Capture | Column-Level Security | Cost Optimization | Data Capture | Data Factory | Data Governance | Data Lake Storage Gen2 | Data Lineage | Data Modeling | Data Vault | Databricks | Databricks SQL | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | ELT | ETL | Entra ID | Git | GitHub | Google Cloud | Infrastructure as Code | Key Vault | MLflow | Metadata Management | Microsoft Entra | Microsoft Entra ID | Monitoring | Oracle | Performance Tuning | PostgreSQL | Power BI | PySpark | Python | RBAC | Retrieval-Augmented Generation | Row Level Security | SQL | SQL Server | Scala | Semantic Modeling | Snowflake | Snowflake Schema | Spark SQL | Star Schema | Streaming Architecture | Structured Streaming | Synapse Analytics | Tableau | Terraform | Unity Catalog | Vector Databases | “as-code”
Education
Related jobs
-
Senior Solutions Engineer INR 2000K-4131KApache Spark | Big Data | Big data analytics | Cloud Computing | Data EngineeringSenior-level Full TimeIndia1d ago
-
Senior Solutions Architect INR 2000K-4131KAWS | Apache Cassandra | Apache Spark | Azure | Cloud ComputingHybrid work | Travel opportunitiesSenior-level Full TimeBengaluru, India R1d ago
-
Senior-level Full TimeIndia - Bangalore Office1d ago
-
Principal Engineer, AI Architect INR 2000K-5000KAI Practices | API Integration | AWS | Agent Orchestration | AzureSenior-level Full TimeBengaluru, India1d ago
-
SW Architect - AI Engineering INR 2500K-5000KAPI Development | AWS | Azure | Cloud platform | Cost OptimizationContinuous learning opportunities | Hybrid work opportunities | Supportive, inclusive work environment | Training programsSenior-level Full TimePune - Maharashtra - India1d ago
-
Access Control | Agile | Alerting | Apache Airflow | AuditabilityContinuous professional development | Flexible working | Inclusive and diverse work environmentMid-level Full TimePune, Maharashtra, India2d ago
-
Manager- Data Aggregation INR 1500K-2000KAmazon Redshift | Apache Airflow | Data Governance | Data Modeling | Data PipelinesMid-level Full TimePune2d ago
-
Consultant - Databricks Azure INR 1000K-2700KAccess Control | Azure | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeBangalore, Karnataka, India2d ago
-
AI/ML Architect INR 4800K-6000KData Mining | Data Pipelines | Deep learning | Distributed Systems | Graph AnalysisSenior-level Full TimeDelhi NCR2d ago
-
Senior-level Full TimeBangalore, Karnataka, India2d ago
-
Associate Process Manager INR 1860K-4000KAI Agents | AI Foundry | AI orchestration | APIs | AWS BedrockMid-level Full TimePune, Maharashtra, India2d ago
-
AWS Bedrock | Amazon Web Services | Anthropic Claude | Apache Airflow | Apache KafkaBonuses for technical content and knowledge sharing | Broadband reimbursement | Continuous learning and professional development | Inclusive global team environment | Medical insurance for parentsSenior-level Full TimeIndia2d ago
-
AI Architect INR 2500K-5000KAWS SageMaker | Azure Machine Learning | Cloud Computing | Data Governance | Deep learningCollaborative working environment | Continuous learning | Global organisationSenior-level Full TimePune, MH, India2d ago
-
Lead Software Engineer INR 1500K-3000KAWS Cloud | Apache Airflow | Azure Data | Azure Data Factory | Data FactorySenior-level Full TimeBangalore, India2d ago
-
Lead Software Engineer INR 1500K-3000KAWS Cloud | Airflow | Apache Airflow | Azure Data | Azure Data FactorySenior-level Full TimeBangalore, India2d ago
-
Amazon Bedrock | Artificial Intelligence | BI | Batch Processing | CCPAEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysExecutive-level Full TimeBangalore, India2d ago
-
Data Engineering Developer- Senior Associate INR 1500K-2500KCI/CD | Data Modeling | Data Quality | Data Warehousing | Data pipelineEmployee networks | Flexible work/life support | Paid volunteer daysMid-level Full TimeHyderabad, India2d ago
-
Software Engineering & Development - Assistant Manager INR 1600K-3000KMicrosoft Fabric | Power Automate | Power BI | Python | SQLFlexible work/life support | Inclusive development opportunities | Paid volunteer days | Vibrant employee networksSenior-level Full TimeBangalore, India2d ago
-
Data Engineering Senior Analyst INR 3000K-3500KAWS | Agile | Azure | C# | CI/CDCross-team collaboration | Engage with multiple technological disciplines | Training and learning opportunitiesSenior-level Full TimeBengaluru, Karnātaka, India2d ago
-
Data Architect INR 2500K-4500KAI architecture | Agentic AI architecture | Azure Machine Learning | Azure OpenAI Service | Bot FrameworkSenior-level Full TimeKolkata, KDC1A, India2d ago
-
Data Engineering Senior Analyst INR 3000K-3500KAWS | Agile | Azure | C# | CI/CDCross-team collaboration | Training opportunitiesSenior-level Full TimeIND Bengaluru, India2d ago
-
AVP, Data Architect (L11) INR 3000K-5000KAWS | AWS Glue | AWS Glue Data Catalog | Amazon Bedrock | Amazon RedshiftExecutive-level Full TimeHyderabad IN, India R2d ago
-
Assistant Manager INR 1500K-2000KAccess Control | Agile | Airflow | Automated testing | Batch ProcessingMid-level Full TimePune, Maharashtra, India2d ago
-
AWS | Asynchronous programming | Azure | Claude Code | FastAPISenior-level ContractShaikpet, India R2d ago
-
Agile | Amazon Neptune | CI/CD | Data Governance | Data LineageFlexible working | Inclusive culture | Opportunities for growth | Professional developmentSenior-level Full TimeHyderabad, Telangana, India R2d ago