Hadoop Big Data Developer
Tasks
- Build ETL and ELT workflows
- Design and develop Hadoop data pipelines
- Design data models and storage layouts
- Develop streaming data pipelines
- Document data architectures and runbooks
- Evaluate and adopt big data and cloud technologies
- Implement data governance, lineage, and quality controls
- Lead architecture audits and performance reviews
- Mentor junior engineers
- Optimize Spark and MapReduce performance
- Orchestrate pipelines with workflow engines
- Set up monitoring, alerting, and logging
Perks/Benefits
Skills/Tech-stack
AWS EMR | Airflow | Apache Atlas | Apache Flink | Apache Pig | Apache Spark | BigQuery | CI/CD | Collibra | Databricks | Delta Lake | HBase | HDFS | Hadoop | Hive | Hudi | Iceberg | Infrastructure as Code | Java | Kafka | Kafka Connect | Kubernetes | MapReduce | NoSQL | ORC | Oozie | Parquet | Python | Python Scripting | SQL | Scala | Shell Scripting | Snowflake | Spark Streaming | Sqoop | Trino | “as-code”
Education
Roles
Big Data Engineer | Data Engineer | Developer | Engineer | Hadoop Developer
Related jobs
-
Senior Data Engineer-JT0224 USD 120K-183K.Net Core | .Net Framework | Apache Airflow | Azure | Azure Data401k match | Career growth opportunities | Dental insurance | Employee resource groups | Health insuranceSenior-level Full TimeRemote, United States R2h ago
-
Business Data Engineer USD 140K-170KAPIs | AWS | Data Automation | Data Ingestion | Data PipelinesFlexible working hours | Vacation policyMid-level Full TimeSan Jose, California or Remote R14h ago
-
Senior Software Engineer - Data Platform USD 115K-145KAWS | Apache Airflow | Apache Spark | Azure | CI/CD401k plan | Coaching therapy professional development | Flexible spending account | Flexible vacation policy | Healthcare coverageSenior-level Full TimeUnited States R16h ago
-
Database Engineer, Senior USD 120K-150KClustering | Database debugging | Database performance | Database performance tuning | High AvailabilityOn-call rotationSenior-level Full TimeUSA - Remote, PA, US R18h ago
-
Anthropic API | Asynchronous programming | Database | Docker | LLM APIFlexible schedule | Fully remote | Part-time hours | Performance-based bonusesMid-level FreelanceTexas, United States - Remote R18h ago
-
Asynchronous programming | Docker | LLM API | NoSQL | Node.jsEnglish proficiency support | Flexible schedule | Performance based bonus programs | Remote workMid-level FreelanceNew York, New York, United States … R18h ago
-
Anthropic | Asynchronous programming | Docker | LLM API | NoSQLBonus programs based on quality | Flexible schedule | Performance-based bonuses | Remote workMid-level FreelanceMichigan, United States - Remote R18h ago
-
Anthropic | Asynchronous programming | Business API | Conversational Design | DatabasesFlexible schedule | Performance bonus programs | Remote workMid-level FreelanceUnited States - Remote R18h ago
-
Anthropic API | Asynchronous programming | Business API | Conversational AI | Discord APIFlexible schedule | Fully remote | Performance-based bonusesMid-level FreelanceSouth Carolina, United States - Remote R18h ago
-
Data Engineer USD 83K-158KAngular | Continuous Delivery | Continuous integration | Data Visualization | Data WarehousingMid-level Full TimeTwo Destiny Way, Westlake TX, United … R18h ago
-
Analytics Engineer, GFCO Analytics USD 152K-179KApache Airflow | Business Intelligence | DBT | Data Governance | Data ModelingQuarterly in person working sessions | Remote-first work environmentMid-level Full TimeRemote - USA R18h ago
-
Data Scientist / ML Engineer USD 170K-210KAWS | Azure | Bias Evaluation | Cloud Computing | Cloud platformFlexible working hours | Remote workSenior-level Full TimeNew York, NY, US, Remote R20h ago
-
Software Engineer, Data Engineering USD 153K-196KC# | C++ | Data Quality | Data pipeline | ETLCross-functional collaboration | Mentorship | OwnershipEntry-level Full TimeSan Mateo, CA, United States R21h ago
-
Machine Learning Engineer USD 138K-183KAWS | Amazon Redshift | Amazon SageMaker | Apache Airflow | Apache FlinkCorporate Bonus Plan | Equity plan | Generous time off | Healthcare | Paid personal time offMid-level Full TimeRemote - US R21h ago
-
Senior Data Platform Engineer USD 135K-180KCI/CD | Coalesce | Dagster | Data Modeling | Data pipeline401k match | Dental insurance | Flexible spending account | Flexible time off | Health insuranceSenior-level Full TimeHybrid in Texas R22h ago
-
Data Solutions Engineer - Hybrid/Durham,NC USD 120K-153KADLS Gen2 | Alerting | Azure | Azure Data | Azure Data Factory401k matching | Employee referral program | Flexible spending account | Health savings account | Medical/dental/vision/life insuranceEntry-level Full TimeDurham, North Carolina, United States R22h ago
-
A/B | A/B Testing | Apache Spark | B testing | CalibrationCommuter benefits | Dental insurance | Disability insurance | Healthcare | Hybrid work scheduleSenior-level Full TimeRedwood City, US R23h ago
-
AWS | Code review | Data Storage | Distributed Systems | KotlinEmployee stock purchase plan | Flexible spending wallets | Remote-first | Time offMid-level Full TimeRemote US R1d ago
-
Associate Data Solutions Engineer USD 95K-162KAI workflows | Apache Iceberg | DBT | Data Governance | Data Modeling401k with employer match | Advancement opportunities | Flexible spending account | Health benefits package | Long-term disability insuranceMid-level Full TimeUS-Remote R1d ago
-
Early-Career Network Engineer (RAN Optimization) USD 82K-128K4G | 5G | Automation | C Band | CBRSEducational assistance | Matching gifts | Paid sick time | Paid vacation | Parental leaveMid-level Full TimePlano,Texas,United States R1d ago
-
Artificial Intelligence/Machine Learning Engineer USD 119K-200KAKS | Anomaly Detection | Artificial Intelligence | Azure Data | Azure Data FactoryHybrid work | Remote workSenior-level Full TimeAustin, TX, United States R1d ago
-
Senior Research Data Engineer (US) USD 150K-200KAirflow | Dagster | Data Drift | Data Generation | Data LakeSenior-level Full TimeRemote - US R1d ago
-
Applied AI Engineer - AI Solutions USD 172K-300KAgentic Workflows | Airflow | Apache Spark | Chroma | CrewAIAnnual travel up to 25% | Employee stock options | Hybrid work | Professional developmentMid-level Full TimeNew York City, NY (Hybrid); Redwood … R1d ago
-
Data Engineer-M-F, 9am-6pm Pacific Time zone USD 80K-112KData Governance | Data Mining | Data Pipelines | Data Quality | Data Security401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeRemote, United States R1d ago
-
Data Engineer USD 110K-160KAPI Integration | Agile | Apache Kafka | Apache Spark | Application Performance Monitoring401k match | Family support programs | Fertility assistance | Hybrid work eligibility | Paid HolidaysMid-level Full TimePlano, TX, United States R1d ago