Hadoop Big Data Developer
Tasks
- Build ETL and ELT workflows
- Design big data pipelines on Hadoop
- Design data models and storage layouts
- Develop streaming data pipelines
- Document data architectures and runbooks
- Implement data governance and quality controls
- Integrate streaming and batch systems
- Mentor junior engineers
- Optimize Spark and MapReduce jobs
- Orchestrate workflows with Airflow or Oozie
- Set up monitoring, alerting, and logging
Perks/Benefits
Skills/Tech-stack
AWS EMR | Airflow | Apache Atlas | Apache Flink | Apache Hive | Apache Hudi | Apache Iceberg | Apache Pig | Apache Spark | Azure HDInsight | BigQuery | CI/CD | Collibra | Databricks | Delta Lake | HBase | HDFS | Hadoop | Infrastructure as Code | Kafka | Kubernetes | MapReduce | NoSQL | ORC | Oozie | Parquet | Python | SQL | Shell | Snowflake | Spark Streaming | Sqoop | Trino | “as-code”
Education
Related jobs
-
Machine Learning Engineer V USD 231K-382KAWS | Agent Orchestration | Automated testing | Azure | CI/CDBonus eligibility | Disability insurance | Life insurance | Paid parental leave | Paid time offSenior-level Full TimeRemote, United States R9h ago
-
Senior AI Engineer USD 145K-181KAWS | Alerting | Azure | Docker | Embeddings401k match | Commuter benefits | Dental | Healthcare | Remote friendly workplaceSenior-level Full Time3750 Market Street, Philadelphia, PA, United … R1d ago
-
AWS | AWS CDK | Access Control | Airflow | Athena401k plan | Health insurance | Paid Holidays | Paid time off | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R1d ago
-
Applied AI Specialist, Commercial Customer Success USD 105K-142KAPI Integration | Accuracy Monitoring | Automated testing | CRM | Evaluation FrameworksRemote workSenior-level Full TimeRemote - US R1d ago
-
Principal Software Engineer, Data Infrastructure USD 295K-345KAWS | Airflow | Chaos Engineering | Data Catalog | Distributed SystemsEquity compensation | Health benefits | Onsite work flexibilitySenior-level Full TimeSan Mateo, CA, United States R1d ago
-
Data Warehouse Software Engineer I USD 70K-80KApache Airflow | Cloud Composer | Clustering | Data Lakes | Data Marts401k match | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeRemote - United States R1d ago
-
Airflow | Auction design | BigQuery | Budget Optimization | Experimentation401k employer match | Coaching support | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R1d ago
-
Data Warehouse Developer USD 65K-70KAPI Integration | Business Intelligence | Case design | Data Modeling | Data WarehousingDental insurance | Health insurance | Life insurance | Paid Holidays | Paid time offMid-level Full TimeNormal, Illinois R1d ago
-
Software Engineer ll, Data Platform USD 110K-120KAPI | Amazon Athena | Amazon EMR | Amazon S3 | Apache Airflow401k match | Company holidays | Disability insurance | Employee assistance program | Flexible spending accountMid-level Full TimeUnited States R1d ago
-
AI Engineer USD 100K-197KARIMA | Amazon SageMaker | Bias Mitigation | Computer Vision | Deep learningMid-level Full TimeUSA - Remote R1d ago
-
Principal AI Platform Engineer USD 190K-225KACR | API Integration | Alerting | Audit Logging | Azure401k match | Career growth professional development | Employee assistance program | Low-cost medical dental vision | Paid HolidaysSenior-level Full TimeRemote (United States) R1d ago
-
Senior Data Engineer-JT0224 USD 120K-183K.Net Core | .Net Framework | Apache Airflow | Azure | Azure Data401k match | Career growth opportunities | Dental insurance | Employee resource groups | Health insuranceSenior-level Full TimeRemote, United States R1d ago
-
Senior Software Engineer, Data Products USD 165K-235KAPIs | Data Pipelines | Data Transformation | Data Warehousing | DatabricksSenior-level Full TimeRemote - US R1d ago
-
AI Integrations Staff Engineer USD 150K-230KAgent SDK | Artificial Intelligence | Backend Development | Data Modeling | Evaluation Frameworks401k contribution | Company retreats | Dental insurance | Employee referral program | EquitySenior-level Full TimeRemote R1d ago
-
Sr. Software Engineer, Machine Learning, tvScientific USD 155K-320KAWS | Adtech | Bandit Algorithms | Causal Inference | Causal LiftSenior-level Full TimeSan Francisco, CA, US; Remote, US R1d ago
-
Senior GenAI Software Developer USD 112K-179KAPI Development | AWS | AWS Bedrock | Angular | AuthenticationAgile team environment | Remote workSenior-level Full TimeUnited States R1d ago
-
Data Science, Advisor USD 135K-216KAPI | AWS | AWS Bedrock | AWS Glue | Amazon KinesisActive secret clearance | Remote work | Travel as neededSenior-level Full TimeUnited States R1d ago
-
Data Architecture, Senior Advisor USD 146K-234KAWS | Access Control | Azure | CI/CD | Cloud Computing100 percent remote | Active clearance optionSenior-level Full TimeUnited States R1d ago
-
Data Architecture, Lead Associate USD 112K-179KAWS | Airflow | Azure | CI/CD | DBT100 percent remote | Active clearance supportSenior-level Full TimeUnited States R1d ago
-
Data Migration Engineer USD 60K-90KBigQuery | Cloud Storage | Cloud platform | Data Cleansing | Data MappingBirthday Bonus Day | Health and wellness days | Holiday time | Medical, dental, and vision coverage | Remote work flexibilityEntry-level Full TimeChicago, IL, United States R1d ago
-
Business Data Engineer USD 140K-170KAPIs | AWS | Data Automation | Data Ingestion | Data PipelinesFlexible working hours | Vacation policyMid-level Full TimeSan Jose, California or Remote R1d ago
-
Senior Software Engineer - Data Platform USD 115K-145KAWS | Apache Airflow | Apache Spark | Azure | CI/CD401k plan | Coaching therapy professional development | Flexible spending account | Flexible vacation policy | Healthcare coverageSenior-level Full TimeUnited States R2d ago
-
Database Engineer, Senior USD 120K-150KClustering | Database debugging | Database performance | Database performance tuning | High AvailabilityOn-call rotationSenior-level Full TimeUSA - Remote, PA, US R2d ago
-
Anthropic API | Asynchronous programming | Database | Docker | LLM APIFlexible schedule | Fully remote | Part-time hours | Performance-based bonusesMid-level FreelanceTexas, United States - Remote R2d ago
-
Asynchronous programming | Docker | LLM API | NoSQL | Node.jsEnglish proficiency support | Flexible schedule | Performance based bonus programs | Remote workMid-level FreelanceNew York, New York, United States … R2d ago