Hadoop Solutions Developer
Tasks
- Adopt new big data and cloud technologies
- Build streaming data pipelines
- Design big data pipelines on Hadoop
- Develop ETL and ELT workflows
- Document data architectures and runbooks
- Implement data governance, lineage, and quality controls
- Ingest relational file streaming and API data
- Integrate with analytics and operational systems
- Manage data models and storage layouts
- Mentor junior engineers
- Optimize Spark and MapReduce performance
- Orchestrate workflows and dependencies
- Perform performance reviews and architecture audits
- Set up monitoring, alerting, and logging
Perks/Benefits
Skills/Tech-stack
AWS EMR | Airflow | Apache Atlas | Apache Flink | Apache HBase | Apache Hive | Apache Hudi | Apache Iceberg | Apache Kafka | Apache Pig | Apache Spark | Azure HDInsight | BigQuery | CI/CD | Collibra | Data Governance | Data Lineage | Data Quality | Databricks | Delta Lake | HDFS | Hadoop | Idempotency | Infrastructure as Code | Kubernetes | MapReduce | ORC | Oozie | Parquet | Python | SQL | Shell | Snowflake | Spark Streaming | Sqoop | Trino | “as-code”
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R15d ago
-
Explosives Research Experiment Lead USD 146K-222KAerospace Engineering | Bench testing | C++ | Computational modeling | Computer Aided Design401k | Education reimbursement | Flexible schedule | Hybrid work schedule | Relocation assistanceSenior-level Full TimeLivermore, CA, United States R1d ago
-
AI Engineer - ICAM USD 146K-197KAccess Management | Agile | Artificial Intelligence | Generative AI | Hugging Face100 percent remote workMid-level Full TimeUSA MD Home Office (MDHOME), United … R1d ago
-
APIs | AWS | CData | Change Data Capture | Cloud Native401k retirement plan | Adoption Assistance | Dental coverage | Education assistance | Employee Learning and DevelopmentSenior-level Full TimeNEO - Tamer Rashid Building OA, … R1d ago
-
AWS AI/ML Technical Practice Lead USD 155K-205KAWS | Agile | Amazon Bedrock | Amazon Rekognition | Amazon SageMakerSenior-level Full TimeUSA - Remote, United States R1d ago
-
Software Engineer, VP USD 169K-205KAPI Gateway | AWS Lambda | Agile | Amazon Bedrock | Amazon EC2Educational assistance | Health and wellness benefits | Paid Holidays | Paid maternity and parental bonding leave | Paid sick daysExecutive-level Full TimeNew Jersey Office - 210 Hudson … R1d ago
-
Vice President, Data Products & Analytics Engineer USD 184K-237KAWS | Agile | Alteryx | BI | CollibraExecutive-level Full TimeNew Jersey Office - 210 Hudson … R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Generative AI Specialist USD 100K-150KAgent systems | Agentic Workflows | Chunking | Deterministic Evaluation | EmbeddingsSenior-level Full TimeUnited States - Remote R1d ago
-
Large Language Model Specialist USD 100K-150KAdapter-Tuning | Attention Optimization | Benchmarking | DPO | DeepSpeed ZeRODirect W2 position | Remote workSenior-level Full TimeTempe, AZ R1d ago
-
Machine Learning Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Computer Vision | Data Quality | Data quality monitoringSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Infrastructure Storage Engineer USD 100K-150KAnsible | Automation | CSI | Capacity Planning | CephCareer growth | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
ML Security Engineer USD 100K-150KAccess Controls | Access Management | Adversarial ML | Application Security | AuthorizationSenior-level Full TimeUnited States - Remote R1d ago
-
ML Performance Engineer USD 100K-150KBenchmarking | C++ | Continuous batching | DeepSpeed | Distributed TrainingRemote workSenior-level Full TimeTempe, AZ R1d ago
-
Machine Learning Engineer USD 80K-90KDeep learning | Evaluation metrics | Generalization | Language Models | Large Language ModelsSenior-level Full TimeFremont, California R1d ago
-
Senior Software Engineer (Storage) - remote in the US USD 100K-150KArgo CD | ArgoCD | Block Storage | CSI | Cluster APISenior-level Full TimeBoston, MA, United States R1d ago
-
Applied AI Lead USD 140K-180KAPI Integration | Agents | Artificial Intelligence | Assistants | Context engineeringEquity package | Health coverage | Remote-first workplace | Unlimited PTOSenior-level Full TimeRemote - United States or Canada R1d ago
-
Principal Applied AI Engineer USD 185K-245KAgent Frameworks | Agent SDK | Anthropic Agent SDK | Context engineering | EvalsSenior-level Full TimeRedesign Health R1d ago
-
AI Forward Deployed Engineer | $120K-$150K + Hybrid + Equity | AI-Based Outage Intelligence Company A USD 120K-150KAI workflows | APIs | Automation | Backend Development | IntegrationDirect impact on company growth | Equity | Hybrid work | Significant ownership | Unlimited PTOMid-level Full TimeKing of Prussia, PA, United States R1d ago
-
API Design | Agentic AI | Cloud Computing | Data Infrastructure | Deployment401k match | Career advancement | Dental insurance | Flexible time off policy | Life insuranceSenior-level Full TimeDallas, Texas, United States; United States R1d ago
-
Machine Learning Research Engineer USD 200K-330KAWS | Azure | Benchmarking | CUDA | DDP401k match | Dental insurance | Health insurance | Paid time off | Vision insuranceMid-level Full TimeEmeryville, California, United States; Hybrid (2-3 … R1d ago
-
Senior HPC and Quantum Systems Engineer USD 272K-431KAPI Integration | Accelerated computing | Amazon Braket | C++ | CUDASenior-level Full TimeUS, MA, Westford R1d ago
-
Machine Learning Engineer USD 155K-228KAWS | Airflow | Argo CD | Azure | CI/CDGenerous time off | Healthcare | Paid parental leave | Paid personal time off | Paid sick timeMid-level Full TimeRemote - US R1d ago