Lead Hadoop Data Engineer
Hyderabad, Telangana, India
INR 2000K-3000K (estimate) Senior-level Full Time
Tasks
- Build optimize batch and real time data pipelines
- Design scalable Big Data solutions using Hadoop ecosystem
- Develop ETL workflows with Python and SQL
- Handle cluster management and Hadoop administration
- Implement data quality governance security compliance
- Lead code reviews and technical discussions
- Manage on premise Big Data infrastructure for availability performance scalability
- Mentor junior engineers on Big Data development standards
- Optimize query performance for large datasets
- Process data using HDFS Hive Spark MapReduce YARN
- Troubleshoot production data pipeline issues
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Data Warehousing | ETL | HBase | HDFS | Hadoop | Hive | Kafka | Linux | MapReduce | On-premise | On-premise infrastructure | Oozie | Pig | Python | SQL | Shell Scripting | Spark | Sqoop | YARN
Education
N/A
Roles
Data Engineer | Engineer | Hadoop Data Engineer | Lead | Lead Hadoop Data Engineer
Related jobs
-
Genetics Data Engineer INR 1500K-2500KAWS | Azure | BGEN | Bcftools | Biobank trusted research environmentsMid-level Full TimeBangalore, Karnataka, IN, 5600376h ago
-
Azure Data Engineer_Consultant INR 1500K-2000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksMid-level Full TimeHyderabad, HYDERABAD, India6h ago
-
Java Spark Engineer INR 2000K-3500KAWS | Apache Spark | Azure | CI/CD | Cloud platformAgile team culture | Career growth opportunitiesSenior-level Full TimeIndia - Bengaluru7h ago
-
AI Engineer - Tieto Tech Consulting (m/f/d) INR 2200K-4200KAI Builder | AI Search | AI Solution | AI solution testing | Agent OrchestrationHybrid work model | Open culture | Professional growth | Work-life balanceMid-level Full TimeBangalore, India7h ago
-
AWS | Agile | Apache Airflow | Apache Beam | Apache Druid401k match | Career development support | Dental insurance | Education support | Flexible work optionsEntry-level Full TimePune, India7h ago
-
Senior Data Engineer – Microsoft Fabric INR 1500K-2800KAudit Logging | Azure Data | Azure Data Factory | Data Factory | Data LakehouseSenior-level Full TimeChennai, TN, India8h ago
-
Sr Software Engineer(Java/Python , Spark and AWS) INR 3000K-4600KAWS | Apache Spark | Automated testing | CI/CD | Cause analysisCross-functional team projects | Learning Campus | Training coursesSenior-level Full TimeMumbai, India8h ago
-
Senior-level Full Timetelengana, India10h ago
-
Data Engineering and Analytics INR 1500K-2500KAPIs | Alteryx | Azure Data | Azure Data Factory | Azure Data LakeEntry-level Full TimeNoida, UP, IN, 20130110h ago
-
AI Engineer INR 2000K-5000KAWS | Azure | CI/CD | Cloud platform | DockerRelocation to Mumbai | Work from OfficeSenior-level Full TimeMumbai - Worli, Mumbai - Worli, …12h ago
-
Creative AI Engineer INR 500K-1600KAI Observability | Anthropic Claude | Asynchronous processing | Azure OpenAI | CachingSenior-level Full TimeMumbai, Maharashtra, India12h ago
-
Engineer – Marketing Data & AI INR 2200K-3000KAgile | Cloud Native | JavaScript | Microservices | Node.jsFamily-friendly benefits | Flexible career path | Flexible work hours | Hybrid work environment | Learning opportunitiesMid-level Full TimeIN-REM-002|IN-GGN-001, India12h ago
-
Access Management | Agent systems | Agentic AI | Amazon Web Services | Artificial IntelligenceSenior-level Full TimeBengaluru, KA, IN, 56004812h ago
-
Senior-level Full TimeIND - Bengaluru, India12h ago
-
AI Engineer INR 2000K-4500KAI SDK | AWS | Accelerate | Autogen | AzureDisconnect week | Flexible working hours | Health insurance | Learning budget | MacBookSenior-level Full TimeIndia - Remote R12h ago
-
Access | Alteryx | Automation | Cause analysis | Change ManagementSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India12h ago
-
Banking Product Analytics VP INR 4000K-5199KAgentic Workflow | Cloud Computing | Data Governance | Data Pipelines | Data QualityExecutive-level Full TimeGurugram, DLF Downtown, India12h ago
-
Data Engineer (Python, PySpark, Snowflake, Databricks) | Cloud Data Pipelines, Automation & Compliance INR 3000K-4000KAWS | Access Control | Azure | Azure DevOps | BitbucketSenior-level Full TimePune - Hinjewadi (Ascendas), India12h ago
-
API Management | AWS | Access Control | Agile | AzureSenior-level Full TimeMumbai, India12h ago
-
Senior-level Full TimeHyderabad - BSR IT - SEZ …12h ago
-
Data Engineer (Python, PySpark, Snowflake, Cloud) | Scalable Data Pipelines & Cloud Migration Support INR 3000K-4000KAWS | Access Control | Azure | Azure DevOps | CloudFormationSenior-level Full TimePune - Hinjewadi (Ascendas), India12h ago
-
Entry-level Full TimePune, Gera Commerzone SEZ, India12h ago
-
Senior Software Engineer-Pyspark developer INR 2139K-3500K.NET | .Net Core | AJAX | ASP.NET | ASP.NET MVCSenior-level Full Time110380-IND-BENGALURU-INTL BLR Twr-1&2 CARNATION, India12h ago
-
Senior-level Full Timetelengana, India17h ago
-
Strategic Projects Lead, Generative AI INR 2500K-4500KData analytics | Data labeling | Generative AI | Product Management | Project ManagementSenior-level Full TimeIndia18h ago