M02
Tasks
- Conduct performance reviews and coaching
- Design batch and real-time data pipelines
- Drive distributed data processing initiatives
- Ensure delivery of software solutions
- Ensure security and coding standards
- Establish Spark best practices
- Guide Spark architecture and implementation
- Improve engineering productivity through automation
- Lead engineering team
- Manage CI/CD automation
- Optimize Spark workload performance
- Provide technical leadership for data processing systems
- Support hiring onboarding and growth
- Track delivery and operational metrics
- Troubleshoot production Spark issues
Perks/Benefits
Skills/Tech-stack
AWS EMR | AWS Glue | AWS Lambda | AWS Step Functions | Agile | Amazon CloudWatch | Amazon ECS | Amazon EKS | Amazon S3 | Apache Airflow | Apache Hadoop | Apache Hive | Apache Iceberg | Apache Spark | BigQuery | CI/CD | Caching | Cloud Dataproc | Cloud Storage | Dataframes | Datasets | GCP Dataflow | Google Cloud | Google Cloud Dataproc | Google Cloud Storage | Java | Memory Management | Observability | Partitioning | Pub/Sub | Python | Query Tuning | Resource Optimization | SDLC | Scala | Spark SQL | Step Functions | Structured Streaming | Testing
Education
N/A
Related jobs
-
Data Science and AI Manager INR 1500K-2500KAWS | Azure | Data Pipelines | Docker | Feature EngineeringMid-level Full TimeChennai, Tamil Nadu, India10h ago
-
Manager of Software Engineering - Java Full Stack, AI/ML INR 2500K-4000KAI | Agile | Application Resiliency | Automation | CI/CDCoaching & mentoring | Diversity inclusion and respect cultureMid-level Full TimeBengaluru, Karnataka, India10h ago
-
SAP BTP GenAI Manager - US Inv INR 1519K-3000KAI Core | AI Launchpad | CDS | Cloud Application Programming Model | Cloud FoundryMid-level Full TimeBangalore, Karnataka, India15h ago
-
Engineering Manager in Back End - Generative Ai (Gurugram based relocation provided) INR 1000K-1250KAerospike | Agile | Application Security | C# | CI/CDRelocation package providedMid-level Full TimeGurugram17h ago
-
Mid-level Full Time111442-IND-BENGALURU-INTL BLR BLK B3 PETUNIA, India20h ago
-
Director, Data Architecture/Engineering INR 3600K-5000KAPI Integration | Agile | Automated testing | CI/CD | Container OrchestrationComprehensive healthcare | Flexible time off | Hybrid work model | Retirement plan | Tuition reimbursementSenior-level Full TimeM11 - Commers III, 43, 44 … R20h ago
-
Senior Systems Operations Engineer - L2 Support, UNIX, ITIL, SQL queries, Autosys, Automation Scripting, Gen AI INR 2771K-3465KAppDynamics | Autosoys | BlazeMeter | Chaos Monkey | DatabaseSenior-level Full Time111443-IND-HYDERABAD-INTL HYD WF CENTRE BLK B8 …20h ago
-
Manager - Data & Analytics INR 3000K-4000KAnalytics engineering | Automation | Data Modeling | Data platform | PythonFast-paced environment | Great team cultureSenior-level Full TimeChennai, Tamil Nadu, India20h ago
-
Service Delivery Manager – AI/ML and Data INR 1500K-2700KAI Services | AWS | Azure AI | Azure AI Services | Azure DataMid-level Full TimeGDC Financial District, India20h ago
-
Databricks Data Engineer - Manager INR 1500K-2700KApache Spark | Azure | Azure Blob | Azure Blob Storage | Azure DataEmployee networks | Flexible work programs | Hybrid work schedule | Medical insurance | Paid volunteer daysMid-level Full TimeBangalore, India20h ago
-
IN_Manager_Data Science_D&A_Advisory_Hyderabad INR 2000K-4000KApache Airflow | Apache Hadoop | Azure | Azure Data | Azure Data FactoryFlexibility programs | Inclusive benefits | Mentorship | Wellbeing supportSenior-level Full TimeHyderabad - Salarpuria, India20h ago
-
Assistant Manager- AI Engineer with Claude INR 1200K-2500KAWS | Autogen | Azure | Containerization | CrewAIMid-level Full TimeBangalore, Karnataka, India1d ago
-
Manager, Data Science INR 1000K-2000KA/B | A/B Testing | B testing | Bayesian Methods | CI/CDGreat Place to Work certificationMid-level Full TimePune, India1d ago
-
Mid-level Full TimeHyderabad, TS, India1d ago
-
Data Engineering / Sr. Associate Director, Software Engineering INR 1400K-2000KApache Airflow | Apache Beam | Auditability | Batch Processing | CI/CDMid-level Full TimeHyderabad, Telangana, India R1d ago
-
Delivery Lead - AI Engineer INR 4000K-5500KAWS | Apache Spark | Automation | Azure | Cloud platformSenior-level Full TimePune, Maharashtra, India1d ago
-
Manager - Forward Deployed Engineer / AI Engineer INR 1500K-2800KAI Model Deployment | AI model | APIs | Agent Frameworks | AzureMid-level Full TimeBangalore, Karnataka, India1d ago
-
Airflow | Apache Beam | Apache Flink | Asynchronous Messaging | BigQueryMid-level Full TimePune, Maharashtra, India1d ago
-
Airflow | Apache Beam | Apache Flink | BigQuery | CI/CDMid-level Full TimePune, Maharashtra, India1d ago
-
API Design | Airflow | Alerting | Apache Beam | Apache FlinkMid-level Full TimePune, Maharashtra, India1d ago
-
Airflow | Apache Beam | Apache Flink | BigQuery | CI/CDMid-level Full TimePune, Maharashtra, India1d ago
-
Manager - Data Governance & Management INR 1685K-2500KAI Governance | Agile | Bias Mitigation | BigQuery | CollibraMid-level Full TimeChennai, TN, India1d ago
-
AI Engineer - Manager INR 1500K-2500KAPI Development | Agentic AI | Azure | Azure Data | Azure Data FactoryMid-level Full TimePune, Maharashtra, India1d ago
-
Manager Data Integration INR 2520K-4064KApache Airflow | BigQuery | Bigtable | Celigo | Cloud ComposerSenior-level Full TimeBengaluru, KA, India1d ago
-
Director, Senior Engineering (.Net FS + AI/ML) INR 2771K-3500K.NET | API Gateway | AWS | AWS CDK | Amazon BedrockContinuing education program | Continuous learning resources | Family-friendly perks | Financial wellness programs | Generous time offSenior-level Full TimeIN - HYDERABAD SKYVIEW, India1d ago