Machine Learning Engineer, Distributed vLLM
Tasks
- Collaborate with engineering and cross functional teams to deliver deliverables
- Communicate development effort visibility
- Contribute to design development and testing of AI inference features
- Develop KV cache aware routing and scoring algorithms
- Develop and maintain distributed inference infrastructure with Kubernetes
- Develop and test inference optimization algorithms
- Implement system components in Go and or Rust for vLLM integration
- Improve resource utilization fault tolerance and stability of inference stack
- Innovate in inference domain upstream community participation
- Optimize memory utilization and request distribution
- Participate in technical design discussions
- Provide code reviews and technical knowledge sharing
Perks/Benefits
Skills/Tech-stack
API Gateway | Cilium | Distributed Systems | Envoy | GPU Profiling | GPU benchmarking | GRPC | Go | HTTP2 | High Performance | High-Performance Computing | Istio | KV cache | Kubernetes | OpenTelemetry | Performance Computing | Python | Reverse Proxy | Rust | SGLang | System programming | TensorRT-LLM | VLLM
Education
Regions
Countries
States
Cities
Related jobs
-
Senior Machine Learning Engineer USD 198K-287KData Engineering | Fine Tuning | Foundation Models | GenAI | Incident ResponseOn-call rotationSenior-level Full TimeRemote - US R2d ago
-
Senior-level Full TimeRemote, US R2d ago
-
Sr. Staff Machine Learning Engineer, Content Ecosystem USD 227K-469KCausal Inference | Data Quality | Experimentation | Game theory | Language ModelsSenior-level Full TimeSan Francisco, CA, US; Remote, US R2d ago
-
Senior Data Platform Engineer USD 133K-197KAWS | Amazon IAM | Amazon Redshift | Ansible | Apache IcebergDental benefits | Free 1Password account | Generous paid time off | Health benefits | Maternity and Parental Leave Top-UpSenior-level Full TimeRemote (United States | Canada) R2d ago
-
Senior Machine Learning Engineer, Vector Bidding Science USD 148K-229KA/B | A/B Testing | B testing | BigQuery | Control TheoryCommute subsidy | Disability insurance | Employee assistance program | Employee resource groups | Employee stock ownershipSenior-level Full TimeRemote, Washington, USA R2d ago
-
Staff Machine Learning Engineer, Vector Bidding Science USD 172K-266KA/B | A/B Testing | B testing | BigQuery | Control TheoryCommute subsidy | Employee resource groups | Employee stock ownership | Generous vacation | Global employee assistance programSenior-level Full TimeRemote, Washington, USA R2d ago
-
Senior ML Engineer USD 152K-228KAmazon SageMaker | Baseten | Batching | CI/CD | Google Vertex401k matching | Dental insurance | Fertility assistance | Flexible time off | Health insuranceSenior-level Full TimeRemote - US & Canada (see … R2d ago
-
Data Engineer - Assistant Vice President USD 100K-135KAWS | Apache Airflow | Apache Flink | Apache Spark | AzureEmployer-Matched Retirement Plan | Parental leave | Remote work Friday flexibility | Subsidized healthcare | Unlimited paid time offExecutive-level Full TimeSalt Lake City, Utah, United States R2d ago
-
Sr AI Engineer USD 124K-171KAPIs | Agentic Workflows | Event Driven | Event-driven architecture | JavaScriptFlexible time off | Learning and development stipend | Medical, dental, and vision insurance | Mental wellbeing resources | Paid HolidaysSenior-level Full TimeRemote - United States R2d ago
-
Lead Data Engineer - Data Platform Engineering (Hybrid) USD 128K-168KApache Airflow | Apache Hadoop | Apache Spark | BigQuery | Cloud Healthcare APISenior-level Full TimeMelville, Suffolk, United States R2d ago
-
Staff Machine Learning Engineer, Ads Conversion USD 222K-389KA/B | A/B Testing | B testing | Computer Vision | Data MiningSenior-level Full TimeSan Francisco, CA, US; Palo Alto, … R2d ago
-
Senior Solution Engineer Optimization USD 109K-150KAI machine learning | APIs | Advanced Analytics | Algorithms | Analytical problem-solvingHybrid option | Remote work | Travel for customer engagementsSenior-level Full TimeRemote (United States) R2d ago
-
Senior Analytics Engineer USD 135KAmazon Redshift | Anomaly Detection | DBT | Data Modeling | LLMFinancial benefits | Full remote options | Hybrid schedule | Medical benefits | Stock optionsSenior-level Full TimeNew York, New York, United States R2d ago
-
Full-Stack Engineer, AI Data Platform USD 130K-200KAWS DynamoDB | Cassandra | Cloud platform | GCS | Google CloudHybrid workMid-level Full TimeSan Francisco Bay Area R2d ago
-
AWS RDS | Agile | Amazon S3 | Jira | MongoDBPST time zone requirement | Remote workMid-level Full TimeRemote, California R2d ago
-
Azure | Azure AI | Azure Data | Azure Data Factory | Azure DevOpsEU remote work days | Employee assistance program | Employer pension plan | Flexible working hours | JobradSenior-level Full TimeHomeoffice R2d ago
-
Senior Engineer - Ingestion & Streaming Frameworks USD 150K-190KAWS DMS | Airbyte | Amazon Web Services | Apache Airflow | Apache SparkSenior-level Full TimeRemote - United States R2d ago
-
Data Engineer II USD 110K-125KAPI | AWS Glue | AWS Lambda | Alerting | Amazon Redshift4 week paid sabbatical | 401k match | Flexible PTO | Fully remote | Health benefitsMid-level Full TimeRemote, US R2d ago
-
AI-ML Engineer - Finance-Hybrid USD 155K-230KAgentic AI | CI/CD | Cloud Computing | Computer Vision | Data EngineeringContinuing education | Dental insurance | FSA | HSA | Health insuranceSenior-level Full TimeRochester, MN, United States R2d ago
-
AI Inference Engineer USD 110K-270KBenchmarking | C# | C++ | Hugging Face | Hugging Face Transformers401k retirement plan | Commuting support | Company Provided Lunches | Flexible paid time off | Medical, dental, vision plansSenior-level Full TimeBurlingame, California, United States R2d ago
-
DMLSS Business Data Analyst / Data Engineer USD 55K-65KConfluence | Data Governance | Data QA | Data QA/QC | Data QualityFederal clearance support | Remote workMid-level Full TimeRemote, United States R2d ago
-
Data Engineer USD 89K-141KAWS Glue | AWS Lambda | Access Control | Amazon Kinesis | Amazon QuickSightFully remote | Secret clearanceMid-level Full TimeUnited States R2d ago
-
Junior AI Engineer (Open to remote) USD 110K-135KAPI Development | Language Model | Language Model Evaluation | Language Models | Language Processing401k | Dental insurance | Health savings account | Medical insurance | Paid time offEntry-level Full TimeNew York, NY, US, NY 10019 R2d ago
-
Senior Healthcare Data Engineer USD 85K-125KAPI Integration | Access Control | Agile | Amazon Web Services | AzureCareer development | Employee benefits | Flexible work across time zones | Remote work option | Wellbeing supportSenior-level Full TimeRemote, United States R2d ago
-
Senior Data Platform Engineer, Remote USD 135K-180KAWS | AWS Lambda | Access Control | Amazon Aurora | Amazon CloudWatchSenior-level Full TimeUnited States, UNITED STATES, United States R2d ago