Hadoop Big Data Developer
Tasks
- Build ETL/ELT workflows
- Design and develop big data pipelines on Hadoop
- Design data models and storage layouts
- Develop streaming data pipelines
- Document data architectures schemas and runbooks
- Implement data governance, lineage, and quality controls
- Ingest data from relational file streaming and API sources
- Mentor junior engineers
- Optimize Spark and MapReduce jobs
- Orchestrate pipeline workflows
- Set up monitoring, alerting, and logging
Perks/Benefits
- N/A
Skills/Tech-stack
AWS EMR | Airflow | Apache Atlas | Apache Flink | Apache Hive | Apache Pig | Apache Spark | Apache Sqoop | Azure HDInsight | CI/CD | Collibra | Databricks | Delta Lake | Google BigQuery | HBase | HDFS | Hadoop | Hudi | Iceberg | Infrastructure as Code | Java | Kafka | Kubernetes | MapReduce | ORC | Oozie | Parquet | Python | SQL | Scala | Shell | Snowflake | Spark Streaming | Trino | “as-code”
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Roles
Big Data Engineer | Data Engineer | Developer | Engineer | Hadoop Developer
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R5d ago
-
Apache Iceberg | CI/CD | Compaction | Cost Management | DBTEmployee Stock Purchase Plan (ESPP) | Flexible spending wallets | Health care coverage | Paid time off | Remote-first work environmentSenior-level Full TimeRemote US R13h ago
-
Advanced Embedded Software Developer USD 87K-109KADO.NET | ASP.NET | C# | C++ | Debugging401(k) retirement benefits | Career growth | Dental insurance | Educational assistance | Employee assistance programSenior-level Full TimePittsford, NY, United States R14h ago
-
Batch Processing | Bias Mitigation | C++ | Data Processing | Experimentation401k employer match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R16h ago
-
Machine Learning Engineer USD 151K-265KC++ | Cloud infrastructure | Data Analysis | Debugging | Distributed ComputingSenior-level Full TimeSan Jose, United States R16h ago
-
Principal Engineer - Data Platform USD 221K-387KAccess Control | Apache Spark | Audit Logging | CDC | CompactionSenior-level Full TimeSanta Clara, California, United States R17h ago
-
Software Engineer - Managed Kubernetes USD 225K-300KC plus plus | C# | CI/CD | CNI | CSIHybrid work schedule | On site office days 4 per week | Work from home day TuesdayMid-level Full TimeSan Francisco Office (Fremont St) R17h ago
-
Senior Data Engineer USD 155K-186KBigQuery | Cloud platform | DBT | Data Governance | Data Ingestion401k match | Accident insurance | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeSan Francisco, CA R17h ago
-
Machine Learning Engineer II USD 160K-231KAlgorithms | Computer Vision | Data Pipelines | Data analytics | Entity recognitionMid-level Full TimeRemote - USA R18h ago
-
API | Agents | C# | Code Reviews | Generative AICollaborative environment | Comprehensive benefits package | Employee ownership | Flexible workplace | Innovative cultureEntry-level Full TimeGreenwood Village, Colorado, United States R18h ago
-
Data Engineer USD 133K-159KBigQuery | Cloud platform | DBT | Data Governance | Data Modeling401k match | Employee assistance program | Employee stock purchase program | Flexible family planning support | Flexible vacation policyMid-level Full TimeSan Francisco, CA R19h ago
-
Senior Software Engineer - Computational Platform USD 154K-200KAPI Development | AWS | Cloud Networking | Data Modeling | Data Pipelines401k plan | Commuter support | Company-provided laptop | Flexible paid time off | HolidaysSenior-level Full TimeRemote; Watertown, Massachusetts, United States R20h ago
-
Senior-level Full TimeDeerfield, IL, United States R20h ago
-
Senior Software Engineer I- Data & Insights Engineering USD 135K-165KAmazon Redshift | BigQuery | CI/CD | Data Pipelines | Database replicationSenior-level Full TimeRemote, United States R21h ago
-
Senior Data Engineer - Databricks USD 180K-247KAWS | Access Control | Apache Spark | Azure | CI/CD401k match | Career development program | Corporate Benefit Program | Educational resources | Employee Referral Bonus ProgramSenior-level Full TimeDenver, CO R22h ago
-
AWS | AWS Config | Checkov | CloudTrail | CloudWatchDomestic travel up to 100 percent | On-call support | Remote workMid-level Full TimeRemote - USA R22h ago
-
Senior Machine Learning Engineer USD 139K-233KAWS | Azure | Benchmarking | Cost modeling | GPU Cost ModelingCareer growth opportunities | Hybrid work model | Learning opportunities | Remote work optionSenior-level Full TimeUSA - Sandy, UT R1d ago
-
Senior-level Full TimeRemote - United States R1d ago
-
Azure Data | Azure Data Factory | Azure DevOps | Azure Synapse | Azure Synapse Analytics401k plan | Company holidays | Floating holidays | Paid time off | Remote workSenior-level Full TimeUnited States R1d ago
-
Staff AI Engineer USD 145K-220KAI Agents | App Development | Backend Services | Distributed Systems | LLM App Development401k match | Employer medical dental vision | Flexible spending account | Paid parental leave | Remote first communitySenior-level Full TimeUnited States (Remote) R1d ago
-
Lead Data Analyst, Marketing USD 120K-150KAffiliate Marketing | Dashboard Standardization | Data Literacy | Data Modeling | LookMLCompetitive benefits | Professional growth path | Profit-sharing bonus | Remote-first work cultureSenior-level Full TimeAsheville, NC (remote); Atlanta, GA (remote); … R1d ago
-
Lead Data Analyst, Marketing USD 120K-150KAffiliate Marketing | Business Intelligence | CAC Analysis | Conversion Rate | Conversion rate optimizationGlobal team | Performance-based growth | Profit-sharing bonus | Remote-first cultureSenior-level Full TimeArlington, VA (remote); Baltimore, MD (remote); … R1d ago
-
Lead Data Analyst, Marketing USD 120K-150KAffiliate Marketing | Conversion Rate | Conversion rate optimization | Dashboard Standardization | Data ModelingHealth benefits | Profit-sharing bonus | Remote-first work cultureSenior-level Full TimeAustin, TX (remote); Dallas, TX (remote); … R1d ago
-
Lead Data Analyst, Marketing USD 120K-150KAffiliate Marketing | Dashboard Development | Data Literacy | Data Modeling | Data WarehousingEquity and inclusion culture | Professional growth path | Remote-first work environmentSenior-level Full TimeChicago, IL (remote); Columbus, OH (remote); … R1d ago
-
Lead Data Analyst, Marketing USD 120K-150KBusiness Intelligence | Dashboard Development | Data Modeling | Data Standardization | Data integrationCareer growth path | Competitive benefits | Profit-sharing bonus | Remote-first teamSenior-level Full TimeLos Angeles, CA (remote); Portland, OR … R1d ago