Lead Hadoop Data Engineer
Hyderabad, Telangana, India
INR 2000K-3000K (estimate) Senior-level Full Time
Tasks
- Build optimize batch and real time data pipelines
- Design scalable Big Data solutions using Hadoop ecosystem
- Develop ETL workflows with Python and SQL
- Handle cluster management and Hadoop administration
- Implement data quality governance security compliance
- Lead code reviews and technical discussions
- Manage on premise Big Data infrastructure for availability performance scalability
- Mentor junior engineers on Big Data development standards
- Optimize query performance for large datasets
- Process data using HDFS Hive Spark MapReduce YARN
- Troubleshoot production data pipeline issues
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Data Warehousing | ETL | HBase | HDFS | Hadoop | Hive | Kafka | Linux | MapReduce | On-premise | On-premise infrastructure | Oozie | Pig | Python | SQL | Shell Scripting | Spark | Sqoop | YARN
Education
N/A
Roles
Data Engineer | Engineer | Hadoop Data Engineer | Lead | Lead Hadoop Data Engineer
Related jobs
-
Genetics Data Engineer INR 1500K-2500KAWS | Azure | BGEN | Bcftools | Biobank trusted research environmentsMid-level Full TimeBangalore, Karnataka, IN, 5600375h ago
-
Azure Data Engineer_Consultant INR 1500K-2000KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksMid-level Full TimeHyderabad, HYDERABAD, India5h ago
-
Java Spark Engineer INR 2000K-3500KAWS | Apache Spark | Azure | CI/CD | Cloud platformAgile team culture | Career growth opportunitiesSenior-level Full TimeIndia - Bengaluru6h ago
-
AI Engineer - Tieto Tech Consulting (m/f/d) INR 2200K-4200KAI Builder | AI Search | AI Solution | AI solution testing | Agent OrchestrationHybrid work model | Open culture | Professional growth | Work-life balanceMid-level Full TimeBangalore, India6h ago
-
AWS | Agile | Apache Airflow | Apache Beam | Apache Druid401k match | Career development support | Dental insurance | Education support | Flexible work optionsEntry-level Full TimePune, India7h ago
-
Senior Data Engineer – Microsoft Fabric INR 1500K-2800KAudit Logging | Azure Data | Azure Data Factory | Data Factory | Data LakehouseSenior-level Full TimeChennai, TN, India7h ago
-
Sr Software Engineer(Java/Python , Spark and AWS) INR 3000K-4600KAWS | Apache Spark | Automated testing | CI/CD | Cause analysisCross-functional team projects | Learning Campus | Training coursesSenior-level Full TimeMumbai, India7h ago
-
Senior-level Full Timetelengana, India9h ago
-
Data Engineering and Analytics INR 1500K-2500KAPIs | Alteryx | Azure Data | Azure Data Factory | Azure Data LakeEntry-level Full TimeNoida, UP, IN, 2013019h ago
-
AI Engineer INR 2000K-5000KAWS | Azure | CI/CD | Cloud platform | DockerRelocation to Mumbai | Work from OfficeSenior-level Full TimeMumbai - Worli, Mumbai - Worli, …11h ago
-
Creative AI Engineer INR 500K-1600KAI Observability | Anthropic Claude | Asynchronous processing | Azure OpenAI | CachingSenior-level Full TimeMumbai, Maharashtra, India11h ago
-
Engineer – Marketing Data & AI INR 2200K-3000KAgile | Cloud Native | JavaScript | Microservices | Node.jsFamily-friendly benefits | Flexible career path | Flexible work hours | Hybrid work environment | Learning opportunitiesMid-level Full TimeIN-REM-002|IN-GGN-001, India11h ago
-
Access Management | Agent systems | Agentic AI | Amazon Web Services | Artificial IntelligenceSenior-level Full TimeBengaluru, KA, IN, 56004811h ago
-
Senior-level Full Timetelengana, India16h ago
-
Strategic Projects Lead, Generative AI INR 2500K-4500KData analytics | Data labeling | Generative AI | Product Management | Project ManagementSenior-level Full TimeIndia17h ago
-
Strategic Projects Lead - Coding INR 2500K-5000KAnalytics | Data Taxonomy | Data analytics | Machine Learning | Operational processSenior-level Full TimeIndia17h ago
-
GenAI Engineer INR 2000K-3465KAPI Design | AWS | AWS Lambda | Azure | CI/CDCompany holidays | Comprehensive health insurance | Employee Wellness Initiatives | Flexible work arrangements | Learning and development opportunitiesSenior-level Full TimeGurugram, Haryana21h ago
-
Senior-level Full TimeHyderabad, India21h ago
-
Engineer - Digital & AI Solutions INR 302K-346KAWS | Azure | Cloud platform | Computer Vision | Deep learningCommunity activities | Discounted gym membership | Flexible working | Life assurance | Medical insuranceEntry-level Full TimeHyderabad, Telangana, India22h ago
-
Copilot Studio / Agentic AI Engineer (Remote) INR 2000K-4800KAI Services | API Integration | Agent Orchestration | Agile | Autonomous workflowsFull-time employment | Remote workSenior-level Full TimeMaharashtra, Pune, India R22h ago
-
Senior-level Full TimeKarnataka, Bengaluru, India22h ago
-
Elastic Engineer INR 2400K-4000KAccess Control | Audit Logging | Dissect | ETL | Elastic AgentDiversity and inclusion focusSenior-level Full TimeMaharashtra, Mumbai, India22h ago
-
C/C++ Embedded SW Specialist INR 3000K-4000KAgile | Bash | C# | C++ | Continuous integrationCollectively agreed terms | Health benefits | Inclusive work environment | Secure employmentSenior-level Full TimePune, MH, India22h ago
-
Senior-level Full TimeBangalore22h ago
-
Data Engineer - AWS/ Databricks INR 1800K-2500KAWS | AWS Glue | AWS Lambda | Amazon EMR | Amazon RedshiftMid-level Full TimeHyderabad, India23h ago