Hadoop Big Data Developer
Tasks
- Automate pipeline orchestration
- Build ETL and ELT workflows
- Deliver curated analytics datasets
- Design big data pipelines on Hadoop
- Design data models and storage layouts
- Develop streaming data pipelines
- Document data architectures and runbooks
- Implement data governance and lineage
- Mentor junior engineers
- Optimize Spark and MapReduce jobs
- Set up monitoring, alerting, and logging
Perks/Benefits
Skills/Tech-stack
Airflow | Apache Flink | Apache Hive | Apache Hudi | Apache Iceberg | Apache Pig | Apache Spark | CI/CD | Delta Lake | HBase | HDFS | Hadoop | Infrastructure as Code | Java | Kafka | Kubernetes | MapReduce | NoSQL | ORC | Oozie | Parquet | Python | SQL | Scala | Spark Streaming | Sqoop | “as-code”
Education
Related jobs
-
Director, Data Science: Data Science Tools USD 125K-211KAirflow | Bash | Celery | Dash | DatabricksHybrid schedule | Professional development | Remote workExecutive-level Full TimeRemote, Remote, United States R9h ago
-
Senior Staff AI Data Engineer - Hybrid USD 135K-202KAWS | Alerting | Azure | Batch pipelines | ChunkingHybrid work schedule | Learning and development opportunities | MentorshipSenior-level Full TimeHartford CT- Home Office, United States R23h ago
-
Mid-level Full TimeUnited States - Remote R23h ago
-
AI Engineer - Application Development USD 75K-158KAPI Integration | AWS | Agno | Amazon Bedrock | AzureFlexible time off | Learning resourcesMid-level Full Time999 REMOTE, United States R23h ago
-
Embedded AI Engineer USD 100K-150KC++ | Core ML | Edge AI | Edge inference | Energy EfficiencyCareer growthSenior-level Full TimeUnited States - Remote R23h ago
-
Senior-level Full TimeUnited States - Remote R23h ago
-
Senior-level Full TimeUnited States - Remote R23h ago
-
Edge AI Engineer USD 100K-150KC++ | Core ML | Energy Efficiency | Federated Learning | Hybrid Cloud100 percent remote | Diversity and inclusion | Long-term career growthSenior-level Full TimeUnited States - Remote R23h ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerators | Computer Vision | Data Quality | Data labeling100 percent remote work | Benefits packageMid-level Full TimeUnited States - Remote R23h ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerators | Baseline Models | Computer Vision | Data QualityFull-time benefits | Remote workMid-level Full TimeUnited States - Remote R23h ago
-
Lead Full Stack Engineer, AI Solutions USD 129K-177KAgentic AI | Angular | Artificial Intelligence | Automated testing | Azure401k retirement savings | Hybrid work option | Life insurance | Long-term disability | Medical/Dental/Vision insuranceSenior-level Full TimeWork at Home - Kentucky, United … R23h ago
-
Senior-level Full TimeUnited States - Remote R23h ago
-
Hadoop Big Data Developer USD 100K-150KAWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HiveH1B transfer support | Remote workSenior-level Full TimeUnited States - Remote R23h ago
-
AI Data Engineer USD 100K-150KApache Beam | CI/CD | Data Lineage | Data Modeling | Data PrivacyCareer growth | Health benefits | Remote workMid-level Full TimeUnited States - Remote R23h ago
-
Senior-level Full TimeUnited States - Remote R23h ago
-
Manager, Data Engineering USD 111K-185KADR | AWS | Analytics infrastructure | Azure | Bicep401k retirement plan | Bereavement leave | Dental insurance | Health insurance | Hybrid work optionMid-level Full TimeCarmel, IN - 11799 N College … R23h ago
-
AI Data Engineer USD 100K-150KApache Beam | CI/CD | Data Lineage | Data Modeling | Data PrivacyFull-time benefits | No third party employment | Remote workMid-level Full TimeUnited States - Remote R23h ago
-
Data Platform Architect USD 100K-150KAccess Control | Access Management | Alation | Amazon Redshift | Apache FlinkFull time W2 employment benefits | Long-term career growth | Remote workSenior-level Full TimeUnited States - Remote R23h ago
-
Senior-level Full TimeUnited States - Remote R23h ago
-
Generative AI Specialist USD 100K-150KAgent Frameworks | Agentic Workflows | Embeddings | Evaluation | Language ModelsCareer growth | Remote workMid-level Full TimeUnited States - Remote R23h ago
-
LLM Engineer USD 100K-150KAdapter Layers | Attention Optimization | Benchmarks | DPO | Data GenerationRemote workMid-level Full TimeUnited States - Remote R23h ago
-
LLM Engineer USD 100KAdapters | Benchmarking | DPO | Dataset curation | Distributed TrainingRemote workMid-level Full TimeUnited States - Remote R23h ago
-
LLM Engineer USD 100K-150KAdapter-Tuning | Benchmarking | DPO | DeepSpeed ZeRO | Evaluation methodologyMid-level Full TimeUnited States - Remote R23h ago
-
Mid-level Full TimeUnited States - Remote R23h ago
-
Machine Learning Research Engineer USD 100K-150KAccelerator hardware | Agentic Systems | Computer Vision | Data Quality | Data labelingRemote work opportunity | W2 employmentSenior-level Full TimeUnited States - Remote R23h ago