Senior Manager
INR 1500K-2800K (estimate) Senior-level Full Time
Tasks
- Automate Databricks deployments
- Build Databricks clusters
- Build Delta Lake tables
- Build Delta Live Tables pipelines
- Build ingestion frameworks with Auto Loader
- Build streaming ingestion pipelines
- Build transformation frameworks
- Configure Unity Catalog governance
- Configure autoscaling
- Create Databricks Jobs
- Deploy models with Databricks Model Serving
- Design Databricks workspaces
- Design Medallion Architecture
- Design multi task job orchestration
- Develop batch data pipelines
- Develop feature engineering pipelines
- Develop real-time data pipelines
- Develop reusable pipeline templates
- Enable Photon Engine
- Fine-tune LLMs
- Implement CI/CD pipelines
- Implement Change Data Feed
- Implement Delta Lake optimization
- Implement RAG solutions
- Implement broadcast joins
- Implement caching strategies
- Implement column masking
- Implement data quality expectations
- Implement dead letter queues
- Implement error handling and retries
- Implement infrastructure-as-code
- Implement row level security
- Implement secrets management
- Implement time travel
- Implement vector search
- Integrate with Airbyte
- Integrate with Fivetran
- Integrate with dbt
- Maintain MLflow model registry
- Maintain compute pools
- Manage MLflow tracking
- Manage data lineage
- Manage schema evolution
- Manage spot instances
- Monitor cloud costs
- Monitor pipeline execution
- Optimize Pyspark performance
- Optimize cluster configurations
- Orchestrate Databricks workflows
- Prepare architecture documentation and runbooks
- Set up logging and monitoring
- Tune Spark with AQE
- Use COPY INTO
- Use Databricks Feature Store
- Use Terraform
Perks/Benefits
Skills/Tech-stack
AWS Glue | AWS Secrets | AWS Secrets Manager | Adaptive Query | Adaptive query execution | Airbyte | Amazon S3 | Asset bundles | Auto Loader | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | Azure Event | Azure Event Hubs | Azure Key Vault | Azure VNet | Broadcast Join | CI/CD | COPY INTO | Caching | DBT | Data Lake Storage | Databricks | Databricks (asset) | Databricks Asset Bundles | Databricks Feature Store | Databricks Model Serving | Databricks Secrets | Delta Lake | Delta Lake Optimization | Delta Live Tables | Delta Live) | Dynamic Partition Pruning | Event Hubs | Feature Store | Firewall Configuration | Fivetran | GitHub Actions | GitLab CI | Google BigQuery | Great Expectations | IAM | Infrastructure as Code | Kafka | Key Vault | Kinesis | MLflow | Medallion Architecture | Model Registry | Model Serving | Partition Pruning | Photon Engine | Private Endpoints | PySpark | Query Execution | Secrets Manager | Spark SQL | Structured Streaming | Terraform | Unity Catalog | VPC injection | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Senior-level Full TimeBengaluru, Karnataka, India R5h ago
-
Software Development Specialist INR 940K-940KAmazon CloudWatch | Amazon DynamoDB | Amazon S3 | Amazon Web Services | Azure DevOpsMid-level Full TimeBangalore, KA, IN R8h ago
-
Java API & Microservices Engineer / Architect INR 2500K-3500KAPI Testing | Azure | CI/CD | Cassandra | DatadogSenior-level Full TimeBangalore, KA, IN R16h ago
-
Senior-level Full TimeBangalore, KA, IN R16h ago
-
Lead Data Engineer INR 3000K-4000KAPI | Agile | Amazon Web Services | Azure | Azure SynapseHybrid workSenior-level Full TimeIN025 Embassy Manyata Business Park, India R16h ago
-
Senior Data Platform Engineer INR 1500K-3000KARM Templates | AWS | Azure DevOps | Azure DevOps Services | Azure EntraHybrid workSenior-level Full TimeIN025 Embassy Manyata Business Park, India R16h ago
-
Senior DevOps Engineer INR 3000K-4000KAlerting | Apache Spark | BigQuery | CI/CD | DataflowFlexible working | Hybrid workingSenior-level Full TimeHyderabad Knowledge City (LTC), India R16h ago
-
Junior AI Specialist INR 300K-420KAWS | Apache Airflow | DBT | Data Modeling | Deep learningCertification programs | Health insurance | Language courses | Mentorship | Professional developmentEntry-level Full TimePune, India R1d ago
-
Senior-level Full TimeBengaluru East, Karnataka, India R1d ago
-
Staff Data Engineer INR 3242K-4200KCloud Data | Cloud data platform | Continuous integration | Data Engineering | Data QualityFlexible work hours | Mentorship | Remote work optionsSenior-level Full TimeHyderabad, Telangana , India R1d ago
-
AI Observability | Agent evaluation | Artificial Intelligence | CI/CD | Continuous EvaluationSenior-level Full TimeHyderabad, India R1d ago
-
Senior Data Engineer- Remote, India INR 1500K-2600KApache Airflow | Apache Kafka | Asset bundles | Azure | Azure DataRemote work in IndiaSenior-level Full TimeIndia R1d ago
-
Senior Data Engineer - Full Stack INR 1500K-2700KAWS | Alerting | Apache Spark | Azure | CI/CDEducation wellness and lifestyle support | Generous time-off policies | Top shelf benefitsSenior-level Full TimeIndia Remote R1d ago
-
A/B | A/B Testing | API | B testing | DashboardingAnnual bonus scheme | Collaborative work environment | Flexible work environment and schedule | Flexible working hours | Group medical insuranceMid-level Full TimeIndia R1d ago
-
Testing Engineering Specialist Advisor INR 2475K-3380K.NET | AI Search | AWS | Application Insights | AzureSenior-level Full TimeNoida, UP, IN R1d ago
-
Systems Integration Specialist Advisor_Storage Cloud INR 800K-1099KAWS | Access Control | Automation | Block Storage | CI/CDMid-level Full TimeBangalore, KA, IN R1d ago
-
AVP - Cloud Database Automation Engineer INR 2337K-3056KAWS RDS | Ansible | Auto Scaling | Cassandra | Cloud ConfigFlexible work hours | Work from homeExecutive-level Full TimeHyderabad IN, India R1d ago
-
AI Engineer INR 1500K-4000KAI Agents | API Design | Amazon Web Services | Async task queues | AzureRemote workMid-level Full TimeRemote - Pune, Maharashtra, India R1d ago
-
AI Engineer INR 1800K-3500KAI Foundry | AI Search | AWS Bedrock | Autogen | Azure AIHybrid work model | Onsite work at designated location | Remote work flexibilityNone Full TimeHyderabad, Telangana, India R1d ago
-
Software Engineer II, Analytics Operations INR 885K-1500KAWS Glue | Airflow | Amazon Athena | Amazon Redshift | Apache SparkMid-level Full TimeHyderabad, India R1d ago
-
Machine Learning Engineer 3 INR 1500K-3000KAttention | Automated retraining | CI/CD | Contrastive Learning | DDPMid-level Full TimeNoida, India R1d ago
-
Machine Learning Engineer 3 INR 1500K-3000KApache Spark | Attention Mechanism | Automated retraining | CI/CD | CheckpointingMid-level Full TimeBangalore, India R1d ago
-
Machine Learning Engineer INR 1860K-3000KApache Spark | Attention Mechanisms | Automated retraining | CI/CD | Contrastive LearningMid-level Full TimeBangalore, India R1d ago
-
Machine Learning Engineer 4 INR 3242K-5000KApache Spark | Attention | Automated retraining | CI/CD | CheckpointingSenior-level Full TimeNoida, India R1d ago
-
Mid-level Full TimeRemote Pune-Baroda, India R1d ago