Data Architect
INR 2400K-3000K (estimate) Senior-level Full Time
Tasks
- Build Python and PySpark data transformation and validation
- Build secure observable DLT pipelines with expectations and medallion layering
- Configure Unity Catalog for access control audit logging and masking
- Create SQL dashboards for KPIs and pipeline health
- Create modular Databricks functions for transformation PII masking and validation
- Define data requirements and structure for applications
- Develop ETL ELT pipelines with Databricks technologies
- Implement ingestion patterns with Auto Loader checkpointing and schema evolution
- Integrate data access across domains and workspaces with lineage tracking
- Model and design application data structure storage and integration
- Package and deploy pipelines with Databricks Asset Bundles and CI CD
- Prepare Gen AI compatible datasets with vector embeddings and feature store
- Troubleshoot and optimize Databricks jobs for performance cost and SLA
- Use Power BI Tableau or Looker with Databricks via ODBC and JDBC
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | Apache Airflow | Asset bundles | Auto Loader | Azure | CI/CD | Cloud Data | Cloud Data Engineering | DBT | DLT | DLT Expectations | Data Engineering | Data Modeling | Data Warehousing | Data orchestration | Databricks | Databricks (asset) | Databricks Asset Bundles | Databricks Vector Search | Delta Lake | Delta Sharing | Feature Store | GCP | GitHub | GitLab | JDBC | Looker | MLflow | ODBC | PII masking | Performance optimization | Photon | Power BI | PySpark | Python | SQL | Serverless computing | Tableau | Unity Catalog | Vector Search
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Roles
Related jobs
-
Architect, Data Management INR 1500K-2800KAWS | Cloud platform | Data Modeling | Data Quality | Data SecuritySenior-level Full TimeBangalore, India12h ago
-
Data Architect INR 2600K-3380KAgile | Architecture governance | Architecture standards | Behavior-Driven Development | CI/CDDiversity and inclusion | Remote WorkingSenior-level Full Timehyderabad, India22h ago
-
Solution Architect - Data Platform Modernization INR 1500K-2500KAzure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeBangalore, KA, IN22h ago
-
Data Engineering Lead Analyst -= HIH - Evernorth INR 1500K-2500KAWS Glue | AWS RDS | Access Control | Amazon Athena | Amazon RedshiftMentorship | Remote work flexibilitySenior-level Full TimeHIH - Hyderabad, India R22h ago
-
Staff AI Solution Architect INR 2000K-4500KAWS | Artificial Intelligence | Automation | Cloud Computing | Data analyticsRelocation assistance providedSenior-level Full TimeBengaluru, India22h ago
-
Data Architect INR 2400K-3000KApache Airflow | Asset bundles | Auto Loader | Bronze Silver Gold | CI/CDSenior-level Full TimeBengaluru, BDC6F, India22h ago
-
Data Architect INR 3000K-4000KCloud Storage | Data Modeling | Data integration | Databricks | PythonSenior-level Full TimeBengaluru, BDC6F, India22h ago
-
Senior-level Full TimeBengaluru, BDC6F, India22h ago
-
Senior-level Full TimeBengaluru, BDC6F, India22h ago
-
Senior-level Full TimeBengaluru, BDC6F, India22h ago
-
Data Architect INR 3000K-4000KCloud Computing | Data Architecture | Data Modeling | Data Pipelines | Data StorageSenior-level Full TimePune, PDC3C, India22h ago
-
Senior-level Full TimeHyderabad, HDC4A, India22h ago
-
Data Architect INR 3000K-4000KCloud Data | Cloud Data Platforms | Data Governance | Data Modeling | Data integrationSenior-level Full TimeIndore, INDC1A, India22h ago
-
Senior-level Full TimeBengaluru, BDC11A, India22h ago
-
Senior-level Full TimeBengaluru, BDC11A, India22h ago
-
Senior-level Full TimeBengaluru, BDC11A, India22h ago
-
Senior-level Full TimeBengaluru, BDC11A, India22h ago
-
Senior-level Full TimeBengaluru, BDC11A, India22h ago
-
Data Architect INR 2400K-3000KApache Spark | Asset bundles | Auto Loader | CI/CD | DLT ExpectationsSenior-level Full TimeBengaluru, BDC6F, India22h ago
-
Senior-level Full TimeBengaluru, BDC6F, India22h ago
-
Senior-level Full TimeBengaluru, BDC6F, India22h ago
-
Senior-level Full TimeBengaluru, BDC6F, India22h ago
-
Data Architect INR 2400K-3000KAsset bundles | Auto Loader | CI/CD | DLT Expectations | Data ModelingSenior-level Full TimeBengaluru, BDC11A, India22h ago
-
Senior-level Full TimeBengaluru, BDC11A, India22h ago
-
Senior-level Full TimeBengaluru, BDC11A, India22h ago