Sr. Data Architect (Databricks)
INR 1500K-2500K (estimate) Senior-level Full Time
Tasks
- Architect AI and ML platforms using Databricks
- Architect batch and real time streaming data pipelines
- Automate deployment of notebooks and jobs
- Build semantic models for reporting and analytics
- Configure row level and column level security
- Define CI/CD strategies
- Define enterprise data architecture
- Define technical standards and best practices
- Design ETL and ELT frameworks using Apache Spark and Databricks
- Design Medallion Architecture
- Design and optimize Delta Lake
- Design and optimize Delta Live Tables
- Design data pipelines for machine learning workloads
- Design end to end Lakehouse architectures using Databricks
- Design enterprise data models
- Develop scalable secure and highly available data platforms
- Enable ML lifecycle management with MLflow
- Establish metadata management lineage and auditing
- Establish version control and branching strategies
- Implement Unity Catalog for centralized governance
- Implement change data capture and incremental processing
- Implement infrastructure-as-code
- Implement performance and cost optimization
- Lead architecture reviews and design workshops
- Lead cloud data modernization and migration
- Mentor data engineers and technical leads
- Optimize Spark jobs and SQL workloads
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Apache Spark | Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2 | Azure DevOps | Azure Key Vault | Azure Synapse | Azure Synapse Analytics | CI/CD | Change Data Capture | Column-Level Security | Cost Optimization | Data Capture | Data Factory | Data Governance | Data Lake Storage Gen2 | Data Lineage | Data Modeling | Data Vault | Databricks | Databricks SQL | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Dimensional Modeling | ELT | ETL | Entra ID | Git | GitHub | Google Cloud | Infrastructure as Code | Key Vault | MLflow | Metadata Management | Microsoft Entra | Microsoft Entra ID | Monitoring | Oracle | Performance Tuning | PostgreSQL | Power BI | PySpark | Python | RBAC | Retrieval-Augmented Generation | Row Level Security | SQL | SQL Server | Scala | Semantic Modeling | Snowflake | Snowflake Schema | Spark SQL | Star Schema | Streaming Architecture | Structured Streaming | Synapse Analytics | Tableau | Terraform | Unity Catalog | Vector Databases | “as-code”
Education
Related jobs
-
Speech Architect INR 2500K-5000KAgile | Android Automotive | Android Automotive OS | Automatic Speech Recognition | Automotive OSSenior-level Full TimeBangalore, IN10h ago
-
Manager, Data Engineering, Smai INR 1500K-2028KAgentic AI | Apache NiFi | Apache Spark | Cloud Data | Cloud Data WarehouseMid-level Full TimeHyderabad - Phoenix Aquila, India10h ago
-
Technical Architect - (Java,AWS) INR 3500K-4995KAWS | AWS Lambda | AWS Step Functions | Amazon ECS | Amazon EKSFamily wellbeing benefits | Health benefits | Paid time off | Work-life balanceSenior-level Full TimeHyderabad, India22h ago
-
Solution Architect – Data & AI Solutions INR 3500K-5000KABAC | API Gateway | AWS Glue | Amazon Redshift | Amazon S3Senior-level Full TimeIndia1d ago
-
Senior Director, Panel Applications - Data Engineering INR 2520K-3380KAWS | Agile | Azure | Big Data | Cloud ComputingEmployee Assistance Program (EAP) | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeChennai, TN, India1d ago
-
Senior-level Full TimeNoida,Uttar Pradesh,India1d ago
-
Agentic Architect INR 4000K-6000KAI Governance | API Gateway | AWS Lambda | AWS Step Functions | Amazon AuroraSenior-level Full TimeIndia1d ago
-
AWS | Apache Spark | Business Intelligence | Data Architecture | Data GovernanceCareer growth | Continuous learning | Employee recognition | Global team collaboration | Inclusive workplaceSenior-level Full TimeIndia1d ago
-
AWS Glue | AWS Kinesis | Access Control | Amazon Bedrock | Amazon RedshiftRemote-first workSenior-level Full TimeIndia R1d ago
-
AWS | Airbyte | Airflow | Azure | CI/CDCollaboration across teams | Inclusive work environment | Learning opportunities | Professional growth | Remote workSenior-level Full TimeIndia1d ago
-
Lead - AI/ML Engineer (Generative AI Agent Systems) INR 3700K-5000KAI Agent Development Kit | Agent Development Kit | Agent Engine | Autonomous Agents | Cloud platformSenior-level Contract Full TimePune, MH, India1d ago
-
IT - Advanced Analytics and Data Solutions Management INR 2500K-4000KAI machine learning | Acceptance Testing | Big Data | Business Intelligence | Cause analysisHealth and wellbeing programs | Personal and professional growth | Supportive work cultureMid-level Full TimeBangalore,IND, India1d ago
-
Director, Software Engineering, Self-service Operational Analytics and Reporting - ITC INR 3000K-6000KAgile | Alteryx | Apache Spark | CI/CD | DatabricksExecutive-level Full TimeINDIA TECHNOLOGY CENTER, India1d ago
-
Data & AI Solutions manager/analyst INR 3380K-4720KAzure | C++ | Cause analysis | Data Compliance | Data GovernanceIn-person work at least 3 days per weekSenior-level Full TimeBangalore–Embassy Business Hub, India1d ago
-
Alerting | Apache Spark | Azure | Azure Data | Azure Data FactoryFlexibility programmes | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeMumbai Shivaji Park, India1d ago
-
Alerting | Apache Spark | Azure | Azure Data | Azure Data FactoryFlexibility programmes | Inclusive benefits | MentorshipSenior-level Full TimeMumbai Shivaji Park, India1d ago
-
Data Modeler – Workday & People Analytics INR 2700K-4000KCloud Data | Cloud Data Warehouses | Data Modeler | Data Warehouse | Data WarehousesSenior-level Full TimeHyderabad, India1d ago
-
Data Architect INR 2000K-2500KCollibra Data Governance | Data Architecture | Data Governance | Data Modeling | Data StorageSenior-level Full TimeChennai, CDC2A, India1d ago
-
Senior Solutions Consultant INR 2000K-3000KAgile | Apache Airflow | Apache Spark | Azure Data | Azure Data FactorySenior-level Full TimeIndia - Remote R1d ago
-
Senior Resident Solution Architect INR 2000K-3200KAgile | Amazon Web Services | Apache Spark | CI/CD | Cloud platformSenior-level Full TimeIndia - Remote R1d ago
-
Senior Solution Architect INR 2000K-4500KAWS | Apache Spark | Big Data | Cassandra | Cloud platformSenior-level Full TimeBengaluru, Karnataka, India1d ago
-
Tech Lead 1 INR 4000K-5500KAPI Design | Access Control | Data Governance | Data Modeling | Data PipelinesSenior-level Full TimeGurugram, Haryana, India1d ago
-
Tech Lead INR 3500K-5000KAPI Design | Analytics Platforms | Artificial Intelligence | Backend Development | Best practicesSenior-level Full TimeIndia - Remote R1d ago
-
Senior Data Engineering Consultant INR 3000K-4200KAWS | Access reviews | Always On | Always On Availability | Always On Availability GroupsCareer development opportunities | Comprehensive benefits | Inclusion-focused culture | On call readiness rotation | Technical conferences and workshopsSenior-level Full TimePrimary location: Hyderabad, India1d ago
-
Data Engineering Lead - Postgres Sql Dba INR 2771K-3500KAWS RDS | Aurora PostgreSQL | Azure PostgreSQL | CI/CD | Cloud SQLSenior-level Full TimePrimary location: Hyderabad, India1d ago