Databricks & GCP Data Platform Architect
Hyderabad, //TS, India
INR 1500K-2040K (estimate) Senior-level Full Time
Tasks
- Build CI CD pipelines for Databricks notebooks jobs and configs
- Build Delta Lake bronze silver gold data layers
- Build Spark jobs and Databricks notebooks
- Build batch pipelines using Spark and Databricks
- Build streaming pipelines using Spark and Databricks
- Conduct architecture and code reviews
- Configure access control with GCP IAM
- Design end to end Databricks Lakehouse architecture on GCP
- Enable audit logging lineage and data classification
- Implement Databricks workspaces clusters jobs workflows
- Implement Unity Catalog governance
- Implement infrastructure as code with Terraform
- Implement ingestion pipelines from PubSub and Kafka
- Implement ingestion pipelines from databases and enterprise applications
- Mentor data engineers with code level support
- Optimize Spark performance and Databricks costs
- Provide production support and root cause analysis
- Set up VPC and private endpoints
- Set up monitoring alerting and dashboards
- Translate business requirements into implemented solutions
- Troubleshoot pipeline failures and production issues
Perks/Benefits
- N/A
Skills/Tech-stack
Alerting | Apache Spark | Audit Logging | BigQuery | CI/CD | Cloud IAM | Cloud Storage | Cloud platform | Data Governance | Data Lineage | Databricks | Delta Lake | Delta Live Tables | Delta Live) | Git | Google Cloud | Google Cloud IAM | Google Cloud Platform | Google Cloud Storage | Infrastructure as Code | Kafka | MLflow | Monitoring | Private Cloud | PubSub | Terraform | Unity Catalog | Vertex AI | Virtual Private Cloud | “as-code”
Education
N/A
Roles
Architect | Data Engineer | Data Platform | Data Platform Architect | Engineer | Platform Architect
Related jobs
-
Data Engineer (MS) (Remote) INR 2040K-3380KCI/CD | Data Transformation | Data Validation | Date normalization | ETLMentorship opportunities | Professional growth | Remote workSenior-level Full TimeMaharashtra, Pune, India R8h ago
-
AI / LLM Integration Engineer (Remote) INR 1800K-2800KAPI Integration | Anthropic SDK | Async APIs | Benchmarking | Confidence scoring100 percent remote | Collaborative culture | Inclusive team culture | Professional growthMid-level Full TimeMaharashtra, Pune, India R8h ago
-
Azure Databricks INR 1500K-2000KAccess Controls | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeMid-level Full TimeIN-AP-Hyderabad9h ago
-
Mid-level Full TimeIN-KA-Bangalore9h ago
-
Software Engineer - AI/ML with Python INR 2500K-3500KAI | AWS | Airflow | Apache Spark | BazelCollaborative work environment | Professional growth | Training and developmentSenior-level Full TimeBangalore - Karnataka, India - Kalyani …9h ago
-
Mid-level Full TimeBangalore - Karnataka, India - Kalyani …9h ago
-
Business glossary | Data Catalog | Data Contracts | Data Lineage | Data ModelingAsync first | Flexible overlap hours | RemoteSenior-level Full TimePune, Maharashtra, India10h ago
-
Senior Data Engineer INR 3000K-4000KADLS Gen2 | Audit Logging | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimePune, Maharashtra, India10h ago
-
Senior Database Engineer INR 2520K-3380KAerospike | Azure | Azure Cosmos | Azure Cosmos DB | Azure DataOn-call support | Relocation assistance | Rotational shifts 24x7Senior-level Full TimeBengaluru, KA, IN12h ago
-
Lead Software Engineer - Data Engineer INR 1500K-2400KABAC | AWS | AWS Glue | AWS Lambda | AWS Step FunctionsSenior-level Full TimeBengaluru, Karnataka, India12h ago
-
Senior Architect, ML Engineering INR 2500K-4500KAWS | Agentic Workflows | Async Programming | Azure | Circuit BreakersSenior-level Full TimePune, Maharashtra, India12h ago
-
Entry-level Full TimePune, Maharashtra, India13h ago
-
Entry-level Full TimePune, Maharashtra, India13h ago
-
Data Engineer - Senior INR 3000K-3500KAnomaly Detection | Application Insights | Azure | Azure Application Insights | Azure SQLSenior-level Full TimePune, Maharashtra, India13h ago
-
Senior-level Full TimePune, Maharashtra, India15h ago
-
Germany (FS- D&A)- Executive-BLR INR 500K-500KAgile | Azure Data | Azure Data Factory | Azure Databricks | Azure DevOpsHybrid workExecutive-level Full TimeBangalore, Karnataka, India15h ago
-
Cloud Support Engineer – Backup & Disaster Recovery INR 938K-1500KAWS | Backup and Disaster Recovery | Cloud Computing | Cloud Storage | DatabasesMid-level Full TimeBangalore, India15h ago
-
Embedded Linux Engineer - Platform Software INR 980K-1200KALSA | ARM | Asynchronous programming | BLE | BluetoothMid-level Full TimeGurugram, Haryana, India16h ago
-
Mid-level Full TimeChennai, Tamil Nadu, India16h ago
-
Bash | Cloud platform | Data Processing | Docker | GCPAsynchronous culture | Flexible management approachMid-level Full TimeBangalore, India16h ago
-
Senior-level Full TimeIndia16h ago
-
Azure DevOps | Bash | CI/CD | Distributed tracing | DockerLearning-focused culture | MentorshipMid-level Full TimeBangalore, Karnataka, India16h ago
-
Azure DevOps | Bash | Distributed tracing | Docker | GitOpsLearning-focused culture | MentorshipMid-level Full TimeBangalore, Karnataka, India16h ago
-
Azure DevOps | Bash | Docker | GitOps | GrafanaLearning-focused culture | MentorshipMid-level Full TimeBangalore, Karnataka, India16h ago
-
Azure DevOps | Bash | CI/CD | Docker | EBPFLearning-focused culture | MentorshipMid-level Full TimeBangalore, Karnataka, India16h ago