Data Infrastructure Engineer (Hadoop)
New York - New York, United States
R
USD 170K-180K Senior-level Full Time
Tasks
- Administer Linux servers for data platforms
- Administer NoSQL databases
- Build big data platforms
- Deploy configure tune scale Kafka clusters
- Design NoSQL schemas
- Design big data platforms
- Implement infrastructure-as-code
- Integrate CI/CD pipelines
- Integrate Hadoop ecosystem tools
- Maintain big data platforms
- Maintain high availability clusters
- Manage Kafka topics partitions and consumer groups
- Manage disaster recovery and production support
- Modernize enterprise big data platforms
- Perform root cause analysis
- Provision configure monitor Hadoop clusters
- Support Hive Spark and HDFS integrations
- Troubleshoot production performance issues
- Tune NoSQL database performance
Perks/Benefits
- 401k match
- Cell phone subsidy
- Community service opportunities
- Comprehensive healthcare coverage
- Disability insurance
- Employee stock purchase plan
- Equity bonus
- Generous time off
- Holiday Breaks
- Life insurance
- Mental health resources
- Monthly fitness reimbursement
- Parental leave
- Peer to peer recognition platform
- Training opportunities
- Volunteer opportunities
Skills/Tech-stack
Aerospike | Ansible | Apache Hive | Apache Kafka | Apache Spark | Bash | CI/CD | Cassandra | CentOS | Ceph | Cloudera | Containerization | DynamoDB | Enterprise Linux | Grafana | HBase | HDFS | Hadoop | Hortonworks | Infrastructure as Code | Kubernetes | Linux | MapR | MongoDB | Monitoring | MySQL | Networking | NoSQL | PostgreSQL | Prometheus | Puppet | Python | Red Hat | Red Hat Enterprise | Red Hat Enterprise Linux | Redis | Rocky Linux | Security | Shell Scripting | Storage management | Terraform | “as-code”
Education
N/A
Regions
Countries
States
Cities
Related jobs
-
Senior Software Engineer, Data Engineering USD 243K-295KAWS | Apache Airflow | Apache Spark | Azure | Cloud ComposerSenior-level Full TimeSan Mateo, CA, United States R7h ago
-
Senior Backend Engineer, Experimentation USD 182K-263KA/B | A/B Testing | APIs | Alerting | B testingDental insurance | Health insurance | Mental health benefits | Restricted stock units | Vision insuranceSenior-level Full TimeRemote - US R13h ago
-
Biometrics | C++ | Computer Vision | Machine Learning | Multimodal LearningOffice three days per week | Relocation assistanceMid-level Full TimeSan Francisco R13h ago
-
AWS | Alibaba Cloud | Apache Doris | Apache Iceberg | Apache MilvusMid-level Full TimeSunnyvale R13h ago
-
Software Engineer - Backend Platform USD 140K-205KAPI Design | AWS | Cloud platform | Databases | Distributed SystemsComprehensive health coverage | Coworking stipend | Equity | Flexible savings accounts | Remote workMid-level Full TimePalo Alto, California, United States R14h ago
-
AWS | Alibaba Cloud | Apache Doris | Apache Iceberg | BackupMid-level Full TimeSunnyvale R14h ago
-
AI Engineer USD 180K-220KAPI Design | Agentic Workflows | Authorization | Avro | CI/CDCareer Development Discussions and Check ins | Caregiver support | Company sponsored professional development | Flexible vacation | Health, dental, vision insuranceMid-level Full TimeChicago, IL (Hybrid) R14h ago
-
Backend Engineer, AI Engineering: Duo Chat USD 115K-194KAPI Design | GraphQL | Langchain | Langgraph | Language ModelsEmployee stock purchase plan | Equity compensation | Flexible paid time off | Growth and development fund | Parental leaveSenior-level Full TimeRemote, Canada; Remote, United States R16h ago
-
Machine Learning Engineer USD 210K-240KA/B | A/B Testing | B testing | Data Pipelines | Data Processing401k plan | Commuter benefits | E Bike Transportation | Generous PTO | Learning and development stipendMid-level Full TimeSan Francisco, CA (Hybrid) R16h ago
-
Data Engineer USD 130K-150KApache Spark | Azure | Azure DevOps | CI/CD | Data ContractsFree dog walking services | Free on-site gym | Free parking | Hybrid remote work | Medical/Dental/Vision insuranceMid-level Full TimeSeattle, United States R17h ago
-
Data Engineer USD 119K-202KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Hybrid work arrangementMid-level Full TimeAlexandria, Virginia, United States R17h ago
-
Research Software Engineer, AI/ML USD 110K-130KCI/CD | Docker | GPU Computing | Grafana | KubernetesHybrid work arrangement | On-site workEntry-level Full TimeBlacksburg, Virginia, Hybrid R17h ago
-
Data Engineer II USD 130K-150KAWS | Airflow | CI/CD | CloudWatch | Containerization401k matching | Employee equity plan | Fertility and adoption reimbursement | Flexible time off | Health insuranceMid-level Full TimeUnited States R17h ago
-
Senior Algorithms and Simulation Software Engineer USD 95K-145KC++ | CI/CD | Control Systems | Git | Hardware-in-the-loopSenior-level Full TimeDevens, MA R18h ago
-
Machine Learning Engineer USD 162K-170KComputer Vision | Data Pipelines | Deep learning | Experiment design | Machine Learning401k plan | Dental insurance | Flexible work hours | Long-term disability | Medical insuranceMid-level Full TimeRemote, United States R18h ago
-
Staff DevOps Engineer USD 175K-228KActive Directory | Autogen | Azure Active Directory | Azure DevOps | CI/CDDental insurance | Flexible working hours | Growth opportunities | Health insurance | Hybrid optionsSenior-level Full TimeChicago, IL, United States R18h ago
-
Senior Software Engineer USD 175K-220KAPI Design | AWS | Agent systems | Autogen | AzureDental insurance | Flexible working hours | Health insurance | Hybrid options | Vision insuranceSenior-level Full TimeDallas, TX, United States R19h ago
-
Data Engineer USD 204K-259KAccess Control | Access Control Lists | BigQuery | C plus plus | Data GovernanceAnnual bonus | Company benefits program | Equity incentive planSenior-level Full TimeMountain View, California, USA R19h ago
-
Staff Software Engineer USD 175K-225KAPI Design | AWS | Agentic AI | Autogen | AzureDental insurance | Flexible working hours | Health insurance | Hybrid work options | Vision insuranceSenior-level Full TimeDallas, TX, United States R19h ago
-
Senior Machine Learning Engineer, Digital Twin Platform USD 240K-253KAWS | Azure | Computer Vision | Data Pipelines | GCPFlex First work policy | Remote workSenior-level Full TimeUnited States - Remote R19h ago
-
APIs | Anomaly Detection | Classification Pipeline | Evaluation | HuggingFace Transformers100 percent remote | BYOD | Eastern Time overlap | USD paymentSenior-level Full TimeRemote R19h ago
-
Staff Software Engineer (L4) Data Platform USD 171K-213KAWS | Apache Hadoop | Apache Hudi | Apache Kafka | Apache Spark401k retirement account | Health care insurance | Paid parental leave | Paid personal time off | Paid sick timeSenior-level Full TimeRemote - US R19h ago
-
Collibra Engineer / Data Governance USD 140K-155KBPMN | Collibra | Data Catalog | Data Governance | Data Monitoring401k | Dental | Disability insurance | Employee assistance program | Employee discountsMid-level Full TimeWashington, D.C., DC, US R20h ago
-
Staff Machine Learning Engineer USD 176K-308KA2A | Asyncio | Embeddings | FastAPI | Java401k match | ESPP | Family leave | Flexible spending accounts | Flexible time offSenior-level Full TimeSanta Clara, California, United States R20h ago
-
Associate AI Engineer, Finance Transformation USD 145K-167KAPI Integration | Agent Frameworks | Data dashboards | LLM APIs | Language ModelsMid-level Full TimeSan Francisco - CA; New York … R20h ago