Sr. Manager - Data Engineer
Hyderabad, India
R
INR 2771K-4064K (estimate) Senior-level Full Time
Tasks
- Collaborate on Spark Iceberg Hive Hadoop and cloud architecture
- Conduct performance reviews mentor engineers and support hiring and onboarding
- Design batch and real-time data pipelines
- Drive large scale distributed processing initiatives
- Drive performance benchmarking capacity planning and cost optimization
- Ensure Agile SDLC and engineering governance
- Ensure delivery of data processing solutions
- Establish Spark best practices for development optimization and observability
- Implement CI CD automation and testing for Spark workloads
- Improve engineering productivity through automation
- Lead data engineering team
- Monitor and ensure reliability scalability and operational efficiency
- Optimize Spark workloads through partitioning and tuning
- Oversee Spark platform modernization
- Provide technical leadership for Apache Spark systems
- Track delivery quality reliability and operational metrics
- Troubleshoot and resolve Spark production issues
Perks/Benefits
Skills/Tech-stack
AWS CloudWatch | AWS ECS | AWS EKS | AWS EMR | AWS Glue | AWS Lambda | AWS S3 | Agile | Apache Hadoop | Apache Hive | Apache Iceberg | Apache Spark | BigQuery | CI/CD | Cloud Dataflow | Cloud Dataproc | Cloud Storage | Dataframes | Dataset APIs | Google Cloud | Google Cloud Dataflow | Google Cloud Dataproc | Google Cloud Storage | Java | Observability | PubSub | Python | Scala | Spark SQL | Structured Streaming | Testing
Education
N/A
Related jobs
-
ETL,K8, NiFi, Kafka, Prod Support/Senior Consultant Specialist INR 2800K-4200KAIOps | AWS | Agile | Ansible | AzureFlexible working | Professional development opportunitiesSenior-level Full TimeHyderabad, Telangana, India R6h ago
-
Data & AI Delivery Leader INR 3242K-4680KAnalytics | Artificial Intelligence | BigQuery | CI/CD | Cloud DataCertification reimbursement | Employee resource groups | Helping hands fund | Inclusive wellbeing program | Learning and development supportSenior-level Full TimeBangalore IBC Knowledge Park, India R19h ago
-
Senior Data Platform Agentic AI Engineer INR 3500K-5000KAWS Cloud | AWS Cloud Computing | AWS Key Management Service | Amazon Aurora | Amazon Elastic Kubernetes ServiceHybrid work environment | On-call rotationSenior-level Full TimeQ8 - Bangalore, India R19h ago
-
Senior Software Engineer, Data Platform INR 2000K-4200KApache Pinot | Artifactory | Git | Go | JavaSenior-level Full TimeIndia (Remote) R1d ago
-
Data Architect & Delivery Lead INR 3600K-5400KAgile | Airflow | Batch Processing | BigQuery | Canonical DataSenior-level Full TimeHyderabad, India R1d ago
-
21 CFR | 21 CFR Part 11 | ADaM | Base SAS | CDISCHybrid work | Remote workSenior-level Full TimeIndia R1d ago
-
Senior Manager, Data Engineering INR 2829K-4200KAWS S3 | Amazon EC2 | Apache Spark | Data Analysis | Data PipelinesHybrid work | Well-being | Work-life balanceSenior-level Full TimeBangalore-Affluence, India R1d ago
-
Lead Cloud Data Engineer - Manager INR 2771K-4064KAWS | Airflow | Autosys | CICD | Cloud SecurityEmployee networks | Flexible work/life support | Hybrid work schedule | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHyderabad, India R1d ago
-
Manager , Data and Analytics INR 650K-1200KArtificial Intelligence | Deployment | JavaScript | Machine Learning | Object-OrientedMid-level Full TimeBangalore, Karnataka, India R2d ago
-
GCP Data Engineer / Associate Director, Data and Analytics INR 1000K-2000KAgile | Artificial Intelligence | As-a-Service | Automated testing | CI/CDFlexible working | Inclusive work environment | Professional developmentMid-level Full TimeHyderabad, Telangana, India R2d ago
-
Databricks SME INR 1500K-2700KAWS | Airflow | Apache Spark | CI/CD | DBTBike to work | Career progression and mentorship | Educational assistance | Employee assistance programme | Fitness classesSenior-level Full TimeMumbai, MH, India R2d ago
-
AWS Bedrock | AWS CloudFormation | AWS CloudWatch | AWS Glue | AWS LambdaEqual opportunity inclusion and diversity | Professional development | Remote work opportunity | Workplace accommodationsSenior-level Full TimeIndia R2d ago
-
Adstock | Agent systems | Agentic AI | Apache Spark | Bayesian MethodsSenior-level Full TimeHyderabad, India R2d ago
-
AI Product Development | Agile | Amazon Web Services | Azure | Cloud ComputingEducational assistance | Life insurance | Medical insurance | Personal accident insurance | Training coursesMid-level Full TimeIND-HYNB-Hyderabad, India R2d ago
-
AVP, Enterprise Risk Analytics (L10) INR 3300K-4000KAWS | Automation | Python | SAS | SQLFlexible workspaces | Regional engagement hubs | Work from homeExecutive-level Full TimeHyderabad IN, India R2d ago
-
Data Platform Architect INR 1500K-2500KAWS EC2 | AWS IAM | AWS S3 | Amazon Redshift | Apache AirflowComprehensive benefits | Internal mobility | Professional development | Remote work opportunitySenior-level Full TimeIND - Chennai - Cirrus, India R2d ago
-
Data Steward INR 1500K-2500KAWS | Agile | Alteryx | CI/CD | Data ArchitectureHybrid work environmentMid-level Full TimeHyderabad (Office), India R2d ago
-
Solutions Engineer - Data Engineering (Hyderabad) INR 2000K-3000KAWS | Apache Airflow | Azure | CI/CD | Cloud platformHybrid work modelSenior-level Full TimeHyderabad R3d ago
-
Mid-level Full TimeBangalore, Karnataka, India R3d ago
-
Continuous Risk Management, Operational Risk Analytics, and Insurable Risks - Assistant Manager INR 1300K-1700KDAX | Data analytics | GenAI | Machine Learning | Operational RiskFlexible working | Hybrid workingMid-level Full TimeHyderabad Knowledge Park Tower 2, India R3d ago
-
Specialist, Data Science INR 987K-1700KAutomation | Business Intelligence | Data Engineering | Data Modeling | Data VisualizationFlexible work arrangements | Hybrid work arrangementsMid-level Full TimeIND - Maharashtra - Pune (Wework), … R3d ago
-
Manager - Data Engineering KDP INR 1500K-2700KAPIs | Agile | Amazon Web Services | Azure | Azure SynapseHybrid workMid-level Full TimeIN025 Embassy Manyata Business Park, India R3d ago
-
Associate Director DDIT IES - M365 Platform and AI INR 1500K-3000KAgentic artificial intelligence | Agile | Artificial Intelligence | Compliance | Data GovernanceMid-level Full TimeHyderabad (Office), India R3d ago
-
Software Engineer - Manager INR 2000K-3500KAWS | ArgoCD | CI/CD | Cloud Migration | Cloud platformHealth insurance | Hybrid work environment | Online learning platform | Paid time offMid-level Full TimeIND-Pune-Equifax Analytics-PEC, India R3d ago
-
Manager, Software Development Engineering (Cloud Security / SASE / Data Analysis / Threat Intelligence / LLM) INR 3600K-4600KAPI | AWS | Azure | Cloud Architecture | Cloud platformEducation reimbursement | Health plans | Hybrid work model | In-office perks | Parental leaveSenior-level Full TimeMohali, IND R4d ago