Machine Learning Engineer, Distributed vLLM
Tasks
- Collaborate with engineering and cross functional teams to deliver deliverables
- Communicate development effort visibility
- Contribute to design development and testing of AI inference features
- Develop KV cache aware routing and scoring algorithms
- Develop and maintain distributed inference infrastructure with Kubernetes
- Develop and test inference optimization algorithms
- Implement system components in Go and or Rust for vLLM integration
- Improve resource utilization fault tolerance and stability of inference stack
- Innovate in inference domain upstream community participation
- Optimize memory utilization and request distribution
- Participate in technical design discussions
- Provide code reviews and technical knowledge sharing
Perks/Benefits
Skills/Tech-stack
API Gateway | Cilium | Distributed Systems | Envoy | GPU Profiling | GPU benchmarking | GRPC | Go | HTTP2 | High Performance | High-Performance Computing | Istio | KV cache | Kubernetes | OpenTelemetry | Performance Computing | Python | Reverse Proxy | Rust | SGLang | System programming | TensorRT-LLM | VLLM
Education
Regions
Countries
States
Cities
Related jobs
-
C++ | Cloud Computing | Code Reviews | Deployment Automation | Distributed Systems401k match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R8h ago
-
Forward Deployed AI Engineer, West USD 125K-175KAWS | Azure | Docker | GCP | Generative AI401k plan | Dental insurance | Medical insurance | Parental leave | Unlimited paid time offMid-level Full TimeRemote (San Francisco) R13h ago
-
Senior Machine Learning Engineer, Reinforcement Learning USD 150K-250KDomain Randomization | Embedded Systems | Gazebo | Isaac-Gym | Mujoco401k retirement plan | Dental insurance | Employee referral bonus | Flexible PTO | Free lunchSenior-level Full TimeColumbus, Ohio or Remote R13h ago
-
Senior Data Platform Engineer USD 140K-220KApache Hudi | Apache Spark | CI/CD | Delta Lake | Distributed StorageSenior-level Full TimePittsburgh, PA or Remote R14h ago
-
Staff Software Engineer, Data Platform USD 170K-240KAPI Design | Backend Services | Frontend Development | JavaScript | Node.js401k match | Dental insurance | Equity stock options | Health insurance | Learning GrantSenior-level Full TimeRemote - USA R14h ago
-
Senior Engineer - Data Platform USD 148K-201KAirgapped environments | CI/CD | CRD | ConnectRPC | Consistency models401k retirement plan | Conference support | Dental insurance | Disability insurance | Flexible time offSenior-level Full TimeRemote, United States R14h ago
-
Associate Engineer - Data Platform USD 102K-138KAI Assisted Development | Command Line | Command-line Interface | Containerization | Docker401k retirement plan | Conferences travel lodging fees | Dental & vision insurance | Disability insurance | Flexible time offMid-level Full TimeRemote, United States R15h ago
-
Data Science Engineer (Shreveport, LA) USD 37K-40KData Historian | Data Visualization | Data analytics | Excel | Machine Learning401k match | Dental insurance | Disability insurance | Health insurance | Life insuranceMid-level Full TimeAtlanta, GA, United States R17h ago
-
C++ | CUDA | CUDA kernels | Concurrency | Distributed SystemsSenior-level Full TimePittsburgh, PA or Remote R17h ago
-
C++ | CUDA | Data parallelism | GRPC | GoSenior-level Full TimePittsburgh, PA or Remote R17h ago
-
Staff Machine Learning Engineer, Underwriting and Credit USD 276K-415KA/B | A/B Testing | AWS | Airflow | B testingFlexible time off | Medical insurance | Modern family planning | Remote work | Retirement savings plansSenior-level Full TimeBay Area, CA, United States of … R18h ago
-
Principal Research Data Engineer USD 142K-185KAirflow | Analytical processing | ArcGIS | Avro | CI/CDDental | Health care | PTO | Retirement | Sick leaveSenior-level Full TimeSt. Louis, Missouri, US R1d ago
-
Mid-level Full TimeScott AFB, IL, United States R1d ago
-
Machine Learning Engineer (Active Secret Clearance) USD 160K-190KAgile | Asynchronous programming | CI/CD | Data Engineering | Docker401k plan | FSA | Fully remote work | HSA | Hybrid onsite optionMid-level Full TimeRemote; Tacoma, WA R1d ago
-
Cloud Storage | Compute Orchestration | Computer Vision | Data Lineage | Data PipelinesEnd-to-end responsibility | Fast-paced startup environment | High autonomy | Onsite work | OwnershipMid-level Full TimeSan Mateo, CA; Onsite R1d ago
-
Principal Optimization Engineer USD 117K-234KCONOPT | Cloud Computing | Convergence analysis | Discrete Optimization | Fluid modelingHealth care benefits | Hybrid remote option | Paid Holidays | Paid sick days | Paid vacationSenior-level Full TimeCAG10: ALC HQ, 1025 Cobb Place … R1d ago
-
APIs | Agile | Azure | Azure Data | Azure Data FactoryPeriodic travel | Remote work permittedSenior-level Full Time6314 Remote/Teleworker US, United States R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Edge AI Engineer USD 100K-150KC++ | Core ML | Data Privacy | Device deployment | Embedded SystemsCareer growth | Equal opportunity employer | Health benefits | Remote workSenior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Edge AI Engineer USD 100K-150KBenchmarking | C plus plus | Core ML | Device security | Edge inferenceSenior-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer (Applied AI) USD 100K-150KAblation Studies | Accelerators | Computer Vision | Data Quality | Data quality monitoringHealth benefits | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer (Applied AI) USD 100K-150KAblation Studies | Accelerator hardware | Agentic Systems | Computer Vision | Data QualityCareer growth | Remote workMid-level Full TimeUnited States - Remote R1d ago
-
AI Research Engineer (Applied AI) USD 100K-150KComputer Vision | Data Quality | Data labeling | Data quality monitoring | Deep learningCareer growth | Equal opportunity employer | Remote workMid-level Full TimeUnited States - Remote R1d ago