Sr. Data Architect (Databricks)
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Architect AI and ML platforms using Databricks
- Architect batch and real time streaming data pipelines
- Automate deployment of notebooks and jobs
- Build semantic models for reporting and analytics
- Configure row level and column level security
- Define CI/CD strategies
- Define enterprise data architecture
- Define technical standards and best practices
- Design ETL and ELT frameworks using Apache Spark and Databricks
- Design Medallion Architecture
- Design and optimize Delta Lake
- Design and optimize Delta Live Tables
- Design data pipelines for machine learning workloads
- Design end to end Lakehouse architectures using Databricks
- Design enterprise data models
- Develop scalable secure and highly available data platforms
- Enable ML lifecycle management with MLflow
- Establish metadata management lineage and auditing
- Establish version control and branching strategies
- Implement Unity Catalog for centralized governance
- Implement change data capture and incremental processing
- Implement infrastructure-as-code
- Implement performance and cost optimization
- Lead architecture reviews and design workshops
- Lead cloud data modernization and migration
- Mentor data engineers and technical leads
- Optimize Spark jobs and SQL workloads
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure Key Vault | Azure Synapse | Azure Synapse Analytics | CI/CD | Change Data Capture | Column-Level Security | Cost Optimization | Data Capture | Data Factory | Data Governance | Data Lake Storage Gen2 | Data Lineage | Data Modeling | Data Vault | Databricks | Databricks SQL | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | ELT | ETL | Entra ID | Git | GitHub | Google Cloud | Infrastructure as Code | Key Vault | MLflow | Metadata Management | Microsoft Entra | Microsoft Entra ID | Monitoring | Oracle | Performance Tuning | PostgreSQL | Power BI | PySpark | Python | RBAC | Retrieval-Augmented Generation | Row Level Security | SQL | SQL Server | Scala | Semantic Modeling | Snowflake | Snowflake Schema | Spark SQL | Star Schema | Streaming Architecture | Structured Streaming | Synapse Analytics | Tableau | Terraform | Unity Catalog | Vector Databases | “as-code”
Education
Related jobs
-
Data Architect & Delivery Lead INR 3600K-5400KAgile | Airflow | Batch Architecture | BigQuery | Canonical ModelingHybrid workSenior-level Full TimeHyderabad, India R1d ago
-
Azure | Business Intelligence | Data Modeling | Data Warehouse | Dimensional ModelingSenior-level Full TimeNoida, Uttar Pradesh, India2d ago
-
Manager - Data Governance/Data Platform INR 1000K-2000KCI/CD | DBT | Data Governance | Data Modeling | Data SecurityMid-level Full TimeGurugram, HR, India2d ago
-
Senior-level Full TimeNoida,Uttar Pradesh,India2d ago
-
Senior Software Engineer, Data Engineering INR 2000K-3000K.Net Core | ASP.NET MVC | ASP.Net Core | Agile | Automated testingSenior-level Full TimePune, Maharashtra, India2d ago
-
Solutions Architect INR 3000K-4000KADLS | AWS | AWS Data | AWS Data Migration | AWS Data Migration ServicesMentoring and coaching | Ownership culture | Professional development training and mentoring | Remote-first cultureSenior-level Full TimeIndia - Remote R2d ago
-
Senior-level Full TimeIndia, Telangana, Hyderabad; India, Karnataka, Bangalore; …2d ago
-
Associate Manager - Analytics INR 1500K-2000KAnalytics | Dashboarding | Data Governance | Data Quality | Data SecurityMid-level Full TimeAgra (AS02AM01), IN3d ago
-
Principal Digital Architect INR 2500K-5000KAgentic architecture | Artificial Intelligence | CI/CD | Cloud Computing | Data ArchitectureOnsite work 5 days a week | Relocation availableSenior-level Full TimeChennai, Tamil Nadu, India3d ago
-
Success Architect-Tableau INR 2800K-4500KCloud Architecture | Dashboard troubleshooting | Data Analysis | Data Modeling | Data PipelinesSenior-level Full TimeIndia - Hyderabad3d ago
-
Senior Staff Solution Architect - AI & Cloud INR 3000K-5000KAWS | AWS CDK | AWS CloudFormation | AWS Glue | AWS Step FunctionsSenior-level Full TimeIND19-01-Bengaluru-EPIP 122 (Phase II), India R3d ago
-
AI Architect - Modern AI Applications INR 3000K-4680KAgent Orchestration | Agent communication | Agent to Agent | Agent-to-Agent Communication | Agentic AISenior-level Full TimeIN, BANGALORE, India3d ago
-
Sr. Dir , Data and analytics Architecture INR 3200K-4720KAPI Design | AWS | Airflow | Analytics engineering | Cloud ArchitectureHybrid workSenior-level Full TimeIND - Telangana - Hyderabad (HITEC … R3d ago
-
Assoc. Dir ,SAP Data and Analytics Tech Architecture INR 2771K-4720KAPIs | Analytics Cloud | BW/4HANA | Batch data | Business Data CloudSenior-level Full TimeIND - Telangana - Hyderabad (Hitec … R3d ago
-
AWS | Data Lake | Data Modeling | Data Visualization | Data WarehouseFlexible work arrangements | Hybrid work arrangement | Visa sponsorshipMid-level Full TimeIND - Telangana - Hyderabad (Hitec … R3d ago
-
Snowflake Data Lead INR 4000K-4720KAWS | Agile | Analytical Functions | CI/CD | DBTFlexible work arrangements | Internal mobility | Learning and development programs | MentoringSenior-level Full TimePune - Client Location, India3d ago
-
Young Apprentice - C00 - PUNE INR 400K-480KAWS Lambda | Amazon S3 | Apache Spark | Git | Jupyter NotebookMentorship | Professional developmentEntry-level ApprenticeshipLocation(s): Pune, Maharashtra, India3d ago
-
IB CDAO Data Architect INR 2771K-4500KBusiness Requirements | Conceptual data modeling | Data Architecture | Data Dictionary | Data GovernanceSenior-level Full TimePune, Gera Commerzone SEZ, India3d ago
-
Data Architect INR 3000K-4064KAgile | Data Architecture | Data Compliance | Data Governance | Data LineageFlexible working options | Hybrid work environment | Paid parental leave | Retirement benefitsSenior-level Full TimeIN: Pune Yerwada Commerzone Campus, India3d ago
-
Senior-level Full TimeBangalore, KA, IN R3d ago
-
Sr. AI Architect INR 3200K-4680KAPI Banking | APIs | AWS | Access Management | Agent systemsClient-facing role | Hybrid workSenior-level Full TimeBangalore, KA, IN R3d ago
-
Senior-level Full TimeBangalore, KA, IN R3d ago
-
Databricks - Principal Data Engineer INR 2400K-4200KAWS | Access Control | Apache Kafka | Attribute modeling | AzureSenior-level Full TimeHyderabad, TG, IN R3d ago
-
Data Engineer Senior Consultant INR 2800K-4000KData Warehousing | Data pipeline | ETL | Python | SQLFlexible work arrangements | Remote workSenior-level Full TimeRemote or PAN India NTT Office, … R3d ago
-
Principal Digital Architect - Generative AI INR 2500K-5000KAgentic Architectures | CI/CD | Cloud Computing | Data Architecture | Distributed SystemsOnsite work five days per week | Relocation assistanceSenior-level Full TimeBangalore, Karnataka, India3d ago