Hadoop Big Data Developer
Tasks
- Automate pipeline orchestration
- Build streaming data pipelines
- Design data models and storage layouts
- Design end to end big data pipelines on Hadoop
- Develop ETL/ELT workflows
- Document data architectures and operational runbooks
- Evaluate and adopt big data and cloud technologies
- Implement data governance, lineage, and quality controls
- Ingest structured unstructured and streaming data
- Lead performance reviews and architecture audits
- Mentor junior engineers
- Optimize Spark and MapReduce jobs
- Set up monitoring, alerting, and logging
Perks/Benefits
Skills/Tech-stack
Airflow | Apache Flink | Apache Hudi | Apache Spark | Data Governance | Data Lineage | Delta Lake | Distributed Systems | Error Handling | Fault Tolerance | HBase | HDFS | Hadoop | Hive | Iceberg | Idempotency | Kafka | Logging | MapReduce | Monitoring | ORC | Oozie | Parquet | Partitioning | Pig | Python | SQL | Shell Scripting | Spark Streaming | Sqoop
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Big Data Engineer | Data Engineer | Developer | Engineer | Hadoop Developer
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R1d ago
-
Ansible | C# | C++ | CMake | CUDAMentorship | Remote work | Training workshopsMid-level Full TimeBaltimore, MD, United States R9h ago
-
Senior Software Engineer, Data Insights USD 196K-243KAWS VPC | Amazon EKS | Analytics infrastructure | Apache Superset | Data EngineeringSenior-level Full TimeSan Mateo, CA, United States R16h ago
-
Senior Software Engineer, Unsecured Installments USD 160K-223KA/B | A/B Testing | Accessibility | B testing | CI/CD401k match | Dental insurance | Disability insurance | Health insurance | Health savings accountSenior-level Full TimeUnited States | Remote R18h ago
-
Software Engineer, Unsecured Installments USD 136K-196KCI/CD | Cypress | GraphQL | Java | JavaScript401k | Company holidays | Dental insurance | Employee assistance program | Health insuranceSenior-level Full TimeUnited States | Remote R18h ago
-
Machine Learning Engineer USD 120K-150KBERT | CI/CD | Document parsing | Embeddings | Fine Tuning401k matching | Dental insurance | Employee assistance program | Flexible spending account | Health savings accountMid-level Full TimeRemote (United States) R20h ago
-
AI Engineer - AI/ML USD 98K-176KAWS | Access Control | Agentic Workflows | Agile | AuditabilityCareer development | Comprehensive benefits | Inclusion culture | TelecommuteMid-level Full TimeMinnetonka, Minnesota, UNITED STATES; REMOTE R23h ago
-
Senior-level Full TimeUnited States - Remote R23h ago
-
AI Research Engineer USD 100K-150KAblation Studies | Accelerator hardware | Computer Vision | Data Quality | Data labelingMid-level Full TimeUnited States - Remote R23h ago
-
Mid-level Full TimeUnited States - Remote R23h ago
-
LLM Engineer USD 100K-150KAdapter-Tuning | Attention Optimization | Dataset curation | Direct Preference Optimization | Evaluation methodology100% remote | Benefits package | Full-time W2 employment | Long term multi year engagementMid-level Full TimeUnited States - Remote R23h ago
-
Prompt Engineer USD 100K-150KAgentic Workflows | Chunking | Embeddings | Evaluation | IndexingBenefits | Career growth | Remote workMid-level Full TimeUnited States - Remote R23h ago
-
Robotics Software Engineer USD 100K-150KBehavior Trees | C++ | Computer Vision | Concurrent programming | Critical softwareMid-level Full TimeUnited States - Remote R23h ago
-
ML Platform Engineer USD 100K-150KAPI Gateway | Automated rollback | Autoscaling | C++ | CachingCareer growth | Direct W2 employment | H1B transfer support | Long-term employment | Remote workSenior-level Full TimeJersey City, NJ R23h ago
-
Applied AI Engineer - Federal (TS Required) USD 170K-300KAgentic Workflows | Airflow | Apache Spark | Chroma | CrewAIAnnual travel up to 25 percent | Federal clearance supportMid-level Full TimeUnited States (Remote); Washington, D.C. (Remote) R23h ago
-
Senior Software Engineer, Data Platforms USD 185K-215KAWS | BigQuery | Data Ingestion | Data Processing | Distributed Systems401k matching | Annual company offsite event | Career growth opportunities | Commuter benefits | Health benefits optionsSenior-level Full TimePalo Alto, CA R1d ago
-
Business Intelligence Engineer USD 105K-144KCloud Data | Cloud Data Platforms | DAX | Data Modeling | Data performanceFlexible hybrid work model | Health and life insurance | Paid time off | Pension/retirement benefits | Personal/Family Care leavesMid-level Full TimeJersey City, NJ, United States R1d ago
-
Agentic AI Engineer – Google ADK/GCP USD 130K-191KAPI Development | Agent memory | Agent systems | CI/CD | Cloud RunFinancial and income protection benefits | Medical/Dental/Vision | Paid time off (PTO) | Tax-advantaged health care accounts | Wellness time offSenior-level Full TimeAustin, TX R1d ago
-
Associate Software Engineer – Full Stack (AI First) USD 150K-210KAPI Integration | Angular | CD pipelines | CI/CD | CI/CD pipelinesCareer development | Distributed work culture | Learning opportunities | Work from anywhereMid-level Full TimePhoenix, Arizona R1d ago
-
Staff Software Engineer - Data Platform USD 166K-232KAirflow | Automated testing | BigQuery | CI/CD | Cloud platformDental insurance | Flexible time off | In-office workspace | LinkedIn Learning access | Medical insuranceSenior-level Full TimeUnited States (Remote) R1d ago
-
Data Engineer USD 77K-90KAPI Integration | Automation | Batch Processing | CI/CD | CSV401k | Dental insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeRemote, USA, United States R1d ago
-
Software Engineer, AI Developer Experience USD 160K-200KAutomation | Cloud automation | Command Line | Command-line Interface | Data Analysis401k | In-home office setup | Internet stipend | Learning & development support | Medical, dental & vision coverageSenior-level Full TimeUnited States - Remote Opportunity R1d ago
-
Data Scientist USD 110K-130KAWS SageMaker | Automated retraining | Drift Detection | Experiment tracking | Feature Engineering401k company match | Company holidays | Dental insurance | Flexible working hours | Fully remoteMid-level Full TimeAlaska, Arizona, California, Colorado, Connecticut, Delaware, … R1d ago
-
Machine Learning Engineer III - AI Agent Engineer - Digital and Technology Partners - Onsite/Hybrid USD 132K-198KAI Foundry | AI Search | AWS | Agent Orchestration | Agent systemsSenior-level Full TimeUnited States R1d ago
-
Data Modeling, Fraud Analytics Consulting (Remote) USD 104K-174KAWS | Data Analysis | Data Modeling | Fraud Detection | Machine LearningPeriodic travel | Remote work | Variable pay opportunityMid-level Full TimeUnited States, UNITED STATES, United States R1d ago