Hadoop Solutions Developer
Tasks
- Audit pipeline performance and architecture
- Build ETL and ELT workflows
- Design big data pipelines on Hadoop
- Design data models and storage layouts
- Develop streaming data pipelines
- Document data architectures and runbooks
- Implement data governance and lineage
- Integrate data sources and sinks
- Mentor junior engineers
- Optimize Spark and MapReduce jobs
- Orchestrate pipeline workflows
- Set up monitoring, alerting, and logging
Perks/Benefits
- N/A
Skills/Tech-stack
AWS EMR | Airflow | Apache Atlas | Apache Flink | Apache Hive | Apache Hudi | Apache Pig | Apache Spark | Apache Sqoop | Azure HDInsight | BigQuery | CI/CD | Collibra | Databricks | Delta Lake | HBase | HDFS | Hadoop | Iceberg | Infrastructure as Code | Java | Kafka | Kubernetes | MapReduce | ORC | Oozie | Parquet | Python | SQL | Scala | Shell Scripting | Snowflake | Spark Streaming | Trino | “as-code”
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R1d ago
-
Senior Software Engineer, Data Insights USD 196K-243KAWS VPC | Amazon EKS | Analytics infrastructure | Apache Superset | Data EngineeringSenior-level Full TimeSan Mateo, CA, United States R12h ago
-
Senior Software Engineer, Unsecured Installments USD 160K-223KA/B | A/B Testing | Accessibility | B testing | CI/CD401k match | Dental insurance | Disability insurance | Health insurance | Health savings accountSenior-level Full TimeUnited States | Remote R15h ago
-
Software Engineer, Unsecured Installments USD 136K-196KCI/CD | Cypress | GraphQL | Java | JavaScript401k | Company holidays | Dental insurance | Employee assistance program | Health insuranceSenior-level Full TimeUnited States | Remote R15h ago
-
Machine Learning Engineer USD 120K-150KBERT | CI/CD | Document parsing | Embeddings | Fine Tuning401k matching | Dental insurance | Employee assistance program | Flexible spending account | Health savings accountMid-level Full TimeRemote (United States) R16h ago
-
Applied AI Engineer - Federal (TS Required) USD 170K-300KAgentic Workflows | Airflow | Apache Spark | Chroma | CrewAIAnnual travel up to 25 percent | Federal clearance supportMid-level Full TimeUnited States (Remote); Washington, D.C. (Remote) R19h ago
-
Senior Software Engineer, Data Platforms USD 185K-215KAWS | BigQuery | Data Ingestion | Data Processing | Distributed Systems401k matching | Annual company offsite event | Career growth opportunities | Commuter benefits | Health benefits optionsSenior-level Full TimePalo Alto, CA R21h ago
-
Business Intelligence Engineer USD 105K-144KCloud Data | Cloud Data Platforms | DAX | Data Modeling | Data performanceFlexible hybrid work model | Health and life insurance | Paid time off | Pension/retirement benefits | Personal/Family Care leavesMid-level Full TimeJersey City, NJ, United States R21h ago
-
Agentic AI Engineer – Google ADK/GCP USD 130K-191KAPI Development | Agent memory | Agent systems | CI/CD | Cloud RunFinancial and income protection benefits | Medical/Dental/Vision | Paid time off (PTO) | Tax-advantaged health care accounts | Wellness time offSenior-level Full TimeAustin, TX R21h ago
-
Associate Software Engineer – Full Stack (AI First) USD 150K-210KAPI Integration | Angular | CD pipelines | CI/CD | CI/CD pipelinesCareer development | Distributed work culture | Learning opportunities | Work from anywhereMid-level Full TimePhoenix, Arizona R22h ago
-
Associate Software Engineer – Frontend (AI First) USD 130K-190KAccessibility | Accessibility Standards | Angular | Browser Compatibility | CI/CDCareer development | Community building | Distributed work environment | Learning opportunities | Work from anywhereMid-level Full TimePhoenix, Arizona R22h ago
-
Staff Software Engineer - Data Platform USD 166K-232KAirflow | Automated testing | BigQuery | CI/CD | Cloud platformDental insurance | Flexible time off | In-office workspace | LinkedIn Learning access | Medical insuranceSenior-level Full TimeUnited States (Remote) R22h ago
-
Data Engineer USD 77K-90KAPI Integration | Automation | Batch Processing | CI/CD | CSV401k | Dental insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeRemote, USA, United States R22h ago
-
Software Engineer, AI Developer Experience USD 160K-200KAutomation | Cloud automation | Command Line | Command-line Interface | Data Analysis401k | In-home office setup | Internet stipend | Learning & development support | Medical, dental & vision coverageSenior-level Full TimeUnited States - Remote Opportunity R22h ago
-
Data Scientist USD 110K-130KAWS SageMaker | Automated retraining | Drift Detection | Experiment tracking | Feature Engineering401k company match | Company holidays | Dental insurance | Flexible working hours | Fully remoteMid-level Full TimeAlaska, Arizona, California, Colorado, Connecticut, Delaware, … R23h ago
-
Machine Learning Engineer III - AI Agent Engineer - Digital and Technology Partners - Onsite/Hybrid USD 132K-198KAI Foundry | AI Search | AWS | Agent Orchestration | Agent systemsSenior-level Full TimeUnited States R1d ago
-
Data Modeling, Fraud Analytics Consulting (Remote) USD 104K-174KAWS | Data Analysis | Data Modeling | Fraud Detection | Machine LearningPeriodic travel | Remote work | Variable pay opportunityMid-level Full TimeUnited States, UNITED STATES, United States R1d ago
-
AI/ML Engineer USD 137K-197KCrewAI | Distillation | Docker | Edge AI | Explainable AI401k matching | Commuter benefits | Education reimbursement | Holiday leave | Life insuranceSenior-level Full TimeWashington, DC or Fully Remote R1d ago
-
Senior AI Software Engineer USD 145K-200KAWS | Asynchronous programming | Asyncio | Azure | CI/CD401k matching | Commuter benefits plan | Education reimbursement | Holiday leave | Life insuranceSenior-level Full TimeWashington, DC or Fully Remote R1d ago
-
Head of AI Engineering & Enablement USD 224K-256KAPI Integration | Agent Orchestration | Agent systems | Audit trails | Data PrivacyHealthcare benefits | Remote work | Work from home supportExecutive-level Full TimeUnited States - Remote R1d ago
-
Senior Software Engineer: AI Discovery and Posture USD 129K-162KAmazon Web Services | Asset discovery | C# | Cloud API | GoEmployee assistance program | Healthcare insurance | Life insurance | Paid Company Holidays | Pension or retirement matchingSenior-level Full TimeU.S. Remote R1d ago
-
AI Engineer USD 112K-161KAWS | Anomaly Detection | Azure | C++ | Cause analysis401k matching | Educational assistance | Maternity leave | Paid Holidays | Paid sick timeMid-level Full TimePlano,Texas,United States R1d ago
-
Lead Data Engineer USD 160K-193KAPI | Access Control | Amazon Redshift | CI/CD | DBT100% remote | 401k match | Adoption benefits | Employee resource groups | Flexible time offSenior-level Full TimeDeerfield, IL, United States R1d ago
-
Senior Software Engineer, Data & Cloud USD 105K-115K.NET | AWS | Agile | Android | Angular401k match | Company cell phone plan | Dental insurance | Device stipend | Employee assistance programSenior-level Full TimeDallas R1d ago
-
Business Data Engineer USD 140K-170KAPIs | AWS | Amazon Web Services | Data Pipelines | PythonFlexible working hours | Remote work options | Vacation policyMid-level Full TimeRemote US R1d ago