Hadoop Big Data Developer
Tasks
- Build streaming data pipelines
- Design and maintain data models on HDFS Hive HBase
- Design big data pipelines on Hadoop
- Develop ETL/ELT workflows
- Document data architecture and runbooks
- Implement data governance and quality controls
- Integrate datasets for analytics and machine learning
- Mentor junior engineers and improve engineering standards
- Optimize Spark and MapReduce jobs
- Orchestrate data pipelines with workflow engines
- Set up monitoring, alerting, and logging
Perks/Benefits
Skills/Tech-stack
Apache Airflow | Apache Flink | Apache Hudi | Apache Iceberg | Apache Oozie | Apache Pig | Apache Spark | CI/CD | Data Governance | Data Lineage | Data Modeling | Delta Lake | Distributed Systems | HBase | HDFS | Hadoop | Hive | Infrastructure as Code | Java | Kafka | Kubernetes | MapReduce | NoSQL | ORC | Parquet | Python | SQL | Scala | Shell Scripting | Spark Streaming | Sqoop | Trino | “as-code”
Education
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R10h ago
-
AI Engineer USD 112K-161KAWS | Anomaly Detection | Azure | C++ | Cause analysis401k matching | Educational assistance | Maternity leave | Paid Holidays | Paid sick timeMid-level Full TimePlano,Texas,United States R10h ago
-
Assistant Director, Power BI Enablement Lead USD 100K-152KALM | Analysis Services | Azure | Azure Analysis | Azure Analysis ServicesHybrid work | Inclusive culture | Professional development | Remote workSenior-level Full TimeBoston, Massachusetts, United States R12h ago
-
Senior Analytics Engineer USD 87K-161KAutomation | CI/CD | DBT | Data Ingestion | Data Modeling401k | Flexible workplace options | Hybrid work options | Paid time off | Remote work optionsSenior-level Full TimeRemote-FL, United States R1d ago
-
Applied Scientist USD 120K-238KConditional Generation | Data Curation | Diffusion Models | Distributed Training | Image CaptioningEntry-level Full TimeSan Jose, United States R1d ago
-
Senior AI/ML Engineer - Hybrid in Eden Prairie, MN USD 120K-214KAPI | Agentic AI | Artificial Intelligence | Azure | CI/CD401k contribution | Career development opportunities | Comprehensive benefits | Equity stock purchase | Incentive and recognition programsSenior-level Full TimePrimary location: Eden Prairie, Minnesota R1d ago
-
Sr AI/ML Engineer - Remote Nationwide or Hybrid in MN/DC USD 120K-214KASR | Anomaly Detection | Autogen | Azure Cognitive | Azure Cognitive Search401k contribution | Comprehensive benefits package | Equity stock purchase | Flexible hybrid option | Incentive and recognition programsSenior-level Full TimePrimary location: Eden Prairie, Minnesota R1d ago
-
Assembly | Binary Analysis | C++ | Exploit analysis | False PositiveComprehensive wellness programs | Employee networks | Paid parental and adoption leaves | Paid time off | Professional development opportunitiesSenior-level Full TimeUSA VA Remote, United States R1d ago
-
Senior Machine Learning Engineer - Fully Remote! USD 160K-218KAutoml | CD | CI | CI/CD | ClassificationDiscounted Child Care | Employee assistance programs | Employee discounts | Health and wellness programs | Medical, dental, and vision benefitsSenior-level Full TimeWork From Home OR, United States R1d ago
-
Edge AI Engineer USD 100K-150KC++ | Core ML | Device security | Edge inference | Energy optimizationSenior-level Full TimeUnited States - Remote R1d ago
-
Edge AI Engineer USD 100K-150KC++ | Core ML | DSP | Edge inference | Embedded Systems100 percent remote | Career growth potential | H1B transfer support | Long term multi year engagementSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Edge AI Engineer USD 100K-150KBenchmarking | C++ | Core ML | DSP | Device deploymentCareer growth potential | Health benefits | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Computer Vision | Deep learning | Distributed Training100 percent remote work | Equal opportunity employer | Long-term career growthMid-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer USD 100K-150KAccelerator hardware | Computer Vision | Data Quality | Data quality monitoring | Deep learningBenefits package | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer (Applied AI) USD 100K-150KAblation Studies | Accelerator hardware | Agentic Systems | Computer Vision | Data QualityLong term multi year engagement | Remote work | W2 employmentMid-level Full TimeUnited States - Remote R1d ago
-
Postdoctoral Data Analytics Computational Sciences USD 79K-127KAdversarial Networks | Computer Vision | Confocal microscopy | Convolutional Neural Networks | Deep learning401k matching | Dental insurance | Health insurance | Holiday pay | Life insuranceMid-level Full TimeUS026 PA Spring House - 1400 … R1d ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Flink | Apache Hive | Apache HudiCareer growth | Equal opportunity employer | Health benefits | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KAirflow | Apache Atlas | Apache Flink | Apache Hive | Apache PigCareer growth | Full-time W2 employment | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Hadoop Big Data Developer USD 100K-150KApache Airflow | Apache Flink | Apache Oozie | Apache Spark | CI/CDBenefits | Career growth opportunities | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Senior Data Engineer USD 100K-150KAWS | AWS Glue | Airflow | Amazon Redshift | Apache SparkRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Mid-level Full TimeUnited States - Remote R1d ago
-
Mid-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Generative AI Specialist USD 100K-150KAgent architecture | Chunking | Embeddings | Evaluation Frameworks | Fine Tuning100% remote | Career growthMid-level Full TimeUnited States - Remote R1d ago