Hadoop Developer
Tasks
- Automate pipeline orchestration
- Build ETL/ELT workflows
- Build monitoring alerting and logging strategies
- Continuously evaluate and adopt new big data technologies
- Design data models and storage layouts
- Design end to end big data pipelines on Hadoop
- Develop streaming data pipelines
- Document data architectures and runbooks
- Implement data governance, lineage, and quality controls
- Mentor junior engineers
- Optimize Spark and MapReduce jobs
- Partner with data scientists and analysts
- Perform performance reviews and architecture audits
Perks/Benefits
Skills/Tech-stack
AWS EMR | Airflow | Apache Atlas | Apache Flink | Apache Hive | Apache Kafka | Apache Pig | Apache Spark | Apache Sqoop | BigQuery | CI/CD | Collibra | Databricks | Delta Lake | HBase | HDFS | Hadoop | Hudi | Iceberg | Infrastructure as Code | Java | Kubernetes | MapReduce | NoSQL | ORC | Oozie | Parquet | Python | SQL | Scala | Shell | Snowflake | Spark Streaming | Trino | “as-code”
Education
Roles
Big Data Engineer | Data Engineer | Developer | Engineer | Hadoop Developer
Related jobs
-
Data Engineer USD 126K-208KAPI Integration | Airflow | Amazon Web Services | BigQuery | CCPADEI initiatives | Dental benefits | Employee rewards program | Medical benefits | Mental health supportMid-level Full TimeRemote, United States R3h ago
-
Alerting | Ansible | Bash | CI/CD | CephRemote workSenior-level Full TimeUnited States, United States R4h ago
-
Ansible | Bash | CI/CD | CentOS | CephContract-to-hire | No sponsorship | Remote workSenior-level Full TimeUnited States, United States R4h ago
-
Machine Learning Engineer USD 131K-178KAWS | Cassandra | Convolutional Neural Networks | Data Lakes | Data PipelinesMid-level Full TimeRemote, NY, US R6h ago
-
Case design | Data Integrity | Database Testing | Database optimization | Defect ReportingSenior-level Full TimeAustin, TX R7h ago
-
Software Engineer, Machine Learning USD 213K-293KAPI Design | Agent Orchestration | Artificial Intelligence | Bias Mitigation | C++Senior-level Full TimeSunnyvale, CA | Remote, US | … R9h ago
-
Senior AI Data Engineer USD 155K-185KApache Airflow | Apache Spark | Azure Synapse | BigQuery | ClickHouseEmployer paid Medical Dental Vision Insurance | Flexible paid time off | Manager check ins | Paid cell phone and service | Paid parental leaveSenior-level Full TimeRemote - United States R18h ago
-
Senior Staff Software Engineer - Data Platform USD 200K-250KAWS Glue | AWS IAM | Amazon EMR | Amazon S3 | AmundsenDevelopment dollars | Employee stock purchase program | Family-forming benefits | Financial coaching | Flexible time offSenior-level Full TimeRemote, USA R18h ago
-
Senior Staff Software Engineer - Data Platform USD 200K-250KAWS EMR | AWS Glue | AWS IAM | AWS S3 | Apache AirflowDevelopment dollars | Financial coaching | Flexible remote work | Flexible time off | Free therapy sessionsSenior-level Full TimeRemote, USA R18h ago
-
Staff Machine Learning Engineer USD 189K-389KCalibration | Contextual Bandits | Contextual Decisioning | Data Validation | EmbeddingsEquity eligible | In Office 1 Day Per WeekSenior-level Full TimeSan Francisco, CA, US; Remote, US R19h ago
-
Power BI Developer-1 USD 83KData Modeling | Data Warehousing | Power Apps | Power Automate | Power BIMid-level Full TimeRemote Work( USA), United States R20h ago
-
Principal AI/ML Engineer USD 165K-226KC# | C++ | CI/CD | CUDA | Computer Vision401k match | Dental insurance | Health insurance | Life insurance | Paid time offSenior-level Full TimeRemote PA - PA PAR, United … R20h ago
-
APIs | Compliance | Distributed Systems | Enterprise Integration | Generative AIOccasional evening calls | Remote workSenior-level Full TimeRemote - US Based R20h ago
-
AV Safety Engineering Analytics Engineer (GPSSC) USD 160K-246KCI/CD | Dash | Docker | GitHub | JenkinsRemote workMid-level Full TimeWork From Home - United States, … R20h ago
-
Agile | C++ | Deep learning | Distributed Computing | GPU ComputingDiscretionary bonus | Flexible time off | Healthcare | Leave benefits | Retirement benefitsExecutive-level Full TimeNY7 - 50 Hudson Yards, New … R20h ago
-
Staff Backend AI Engineer, Remote USD 140K-215KAPI Gateway | AWS CDK | AWS ECS | AWS EKS | AWS Fargate401k matching | Dental insurance | Flexible time off | Flexible work schedule | Medical insuranceSenior-level Full TimeUnited States, UNITED STATES, United States R21h ago
-
Data Engineer USD 160K-210KAPI Integration | AWS | Amazon Kinesis | Artificial Intelligence | CI/CDSenior-level Full TimeUS - Remote R21h ago
-
Senior Machine Learning Engineer, Agentic USD 163K-245KA/B | A/B Testing | B testing | Collaborative Filtering | Content-Based Filtering401-K matching | Fertility benefits | Health insurance | Life and disability insurance | Mental health benefitsSenior-level Full TimeBellevue, WA; Menlo Park, CA R22h ago
-
Data Engineer (Remote) USD 155K-180KAzure DevOps | Bash | Cloud Storage | Cloud platform | Compute EngineFully remote | W2 employmentMid-level Full TimeRochester, MN R23h ago
-
Senior Data Engineer (AI Native) USD 103K-192KAWS EMR | Amazon Kinesis | Amazon MWAA | Amazon S3 | Apache Airflow401k company match | Employee assistance program | Equipment and tools support | Flexible PTO | Learning and developmentSenior-level Full TimeRemote, USA ; Remote, Canada R1d ago
-
Principal Snowflake Data Platform Engineer USD 165K-216KAWS | Agile | Ansible | Backup and Resiliency | BashFlexible hybrid work model | Health and life insurance | Paid time off | Retirement benefitsSenior-level Full TimeTX, United States R1d ago
-
Senior Data Engineer (Remote) USD 108K-160KAkamai | Apigee hybrid | Argo CD | Azure AKS | Batch ProcessingFully remote | W2 Candidates OnlySenior-level Full TimeRochester, MN R1d ago
-
AWS RDS | Agile | Amazon S3 | Dashboards | Data PipelinesMust reside in PST time zone | Remote workMid-level Full TimeSan Jose, CA R1d ago
-
AWS RDS | Agile | Amazon S3 | Jira | MongoDBMust reside in PST time zone | Remote workMid-level Full Timephoenix, AZ R1d ago
-
AWS RDS | Agile | Amazon S3 | Dashboards | Data PipelinesDocumentation support | PST time zone requirement | Remote workMid-level Full Timeremote, CA R1d ago