Senior Engineer, Inference Data Plane
Tasks
- Architect and refine system design for multi tenant AI inference cloud
- Build Kubernetes native distributed inference frameworks
- Coach and mentor junior engineers
- Contribute upstream to llm d vLLM and inference gateway projects
- Drive end to end design development and delivery of data plane services
- Implement and optimize distributed inference hosting
- Implement prefill decode disaggregation and KV cache aware routing
- Maintain and operate high scale services with observability and SLOs
- Solve inference aware load balancing and flow control across tenants
Perks/Benefits
- Employee assistance program
- Flexible time off
- Hybrid work model
- LinkedIn Learning access
- Local Employee Meetups
- Reimbursement for conferences training and education
Skills/Tech-stack
Autoscaling | Continuous batching | Data parallelism | GRPC | Go | KServe | KV cache | Kubernetes | Llm-d | NVIDIA TensorRT | Paged Attention | Prefix caching | Python | Quantization | Ray Serve | SGLang | Tensor Parallelism | TensorRT-LLM | VLLM
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R14d ago
-
Machine Learning Engineer (all genders) EUR 63K-69KDeep learning | Machine Learning | NumPy | Pandas | PyTorchCorporate benefits discounts | Health and wellbeing | Hybrid work flexibility | Inclusive work environment | Job ticketsEntry-level Full TimeFrankfurt a.M., Hessen, DE3h ago
-
Machine Learning Engineer (alle Geschlechter) EUR 63K-69KData Analysis | Data Pipelines | Deep learning | Machine Learning | Model DevelopmentCoaching and mentoring | Company Childcare Support | Family support programs | Flexible working hours | Health checksEntry-level Full TimeFrankfurt a.M., Hessen, DE3h ago
-
Sr. Manager, Analytics Engineering & BI Architecture USD 137K-206KAWS | Access Control | Azure DevOps | CI/CD | CloudFormationHealth insurance | Hybrid work schedule | Paid time off | Retirement contributionsSenior-level Full TimeBoston, Massachusetts, US, 022105h ago
-
AI orchestration | BOM | Constraint Optimization | Deep research | Digital TwinsDental insurance | Health insurance | Housing subsidy | In-office meals | Relocation supportSenior-level Full TimeSan Jose7h ago
-
Forward Deployed Engineer - Labrynth USD 150K-230KAI evals | API Gateway | API Integration | AWS IAM | Amazon BedrockCompetitive compensation | Direct customer exposure | High-impact work | Remote-firstSenior-level Full TimeUnited States8h ago
-
Forward Deployed Engineer (FDE) - Public Sector USD 152K-210KAWS | Apache Spark | Azure | Big Data | CI/CDTravel 20%Mid-level Full TimeUnited States10h ago
-
Agentic AI Engineer USD 94K-149KCI/CD | Distributed tracing | Docker | Elasticsearch Relevance Engine | IAMFlexible work schedules | Generous vacation days | Health coverage | Parental leave | Volunteer time offSenior-level Full TimeUnited States10h ago
-
Explosives Research Experiment Lead USD 146K-222KAerospace Engineering | Bench testing | C++ | Computational modeling | Computer Aided Design401k | Education reimbursement | Flexible schedule | Hybrid work schedule | Relocation assistanceSenior-level Full TimeLivermore, CA, United States R11h ago
-
Machine Learning Engineer USD 80K-90KDeep learning | Evaluation metrics | Generalization | Language Models | Large Language ModelsSenior-level Full TimeFremont, California R12h ago
-
Embedded Software Engineer, Access Control USD 125K-160KBash | C# | C++ | Device Drivers | Distributed SystemsCommuter benefits | Fertility benefits | Flexible PTO | Flexible spending account | Health Saving AccountMid-level Full TimeSan Mateo, CA United States12h ago
-
Staff AI Data Platform Engineer USD 138K-173KAPI Gateway | AWS | Access Control | Data Lineage | Data QualityEducational reimbursement | Employee insurance plan | Unlimited vacation | Wellness reimbursementSenior-level Full TimeSan Ramon, CA; Reno, NV12h ago
-
Senior Software Engineer - Machine Learning USD 134K-182KApache Flink | Apache Kafka | Big Data | Data Engineering | Deep learning401k match | Dental insurance | Health insurance | Life insurance | Long-term disabilitySenior-level Full TimeFrisco, TX 75034, United States13h ago
-
Senior Software Engineer (Storage) - remote in the US USD 100K-150KArgo CD | ArgoCD | Block Storage | CSI | Cluster APISenior-level Full TimeBoston, MA, United States R13h ago
-
Applied AI Lead USD 140K-180KAPI Integration | Agents | Artificial Intelligence | Assistants | Context engineeringEquity package | Health coverage | Remote-first workplace | Unlimited PTOSenior-level Full TimeRemote - United States or Canada R13h ago
-
Principal Applied AI Engineer USD 185K-245KAgent Frameworks | Agent SDK | Anthropic Agent SDK | Context engineering | EvalsSenior-level Full TimeRedesign Health R13h ago
-
AI Forward Deployed Engineer | $120K-$150K + Hybrid + Equity | AI-Based Outage Intelligence Company A USD 120K-150KAI workflows | APIs | Automation | Backend Development | IntegrationDirect impact on company growth | Equity | Hybrid work | Significant ownership | Unlimited PTOMid-level Full TimeKing of Prussia, PA, United States R13h ago
-
API Design | Agentic AI | Cloud Computing | Data Infrastructure | Deployment401k match | Career advancement | Dental insurance | Flexible time off policy | Life insuranceSenior-level Full TimeDallas, Texas, United States; United States R14h ago
-
Staff Machine Learning Engineer, Foundation - Seattle USD 208K-298KAWS | Accelerate | Azure | C++ | DeepSpeedExperience bonus | Hybrid work model | QGroup communities | Wellness reimbursementSenior-level Full TimeSeattle, Washington, United States14h ago
-
Active Learning | DPO | Data Privacy | Deep learning | Few-Shot Learning401k match | Family leave | Flexible time away | Health plansSenior-level Full TimeMountain View, CALIFORNIA, United States14h ago
-
Machine Learning Engineer, Agentic Application USD 139K-216KAPI Design | Distributed Systems | Generative AI | Language Models | Language Processing401k match | ESPP | Family leave | Flexible spending account | Flexible time offEntry-level Full TimeMountain View, CALIFORNIA, United States14h ago
-
C++ | DistributedTraining | ETL | Go | Huggingface401k matching | ESPP | Family leave | Flexible spending accounts | Flexible time awaySenior-level Full TimeMountain View, CALIFORNIA, United States14h ago
-
C++ | Distributed Training | ETL Pipelines | Go | Hugging Face401k plan | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeMountain View, CALIFORNIA, United States14h ago
-
Abstractive summarization | Active Learning | DPO | Few-Shot Learning | Few-shotFlexible work persona | Remote work flexibilitySenior-level Full TimeMountain View, CALIFORNIA, United States14h ago
-
Embodied AI Engineer USD 150K-210KC++ | Computer Vision | Data Pipelines | Deep learning | Depth Imaging401k matching | Company holidays | Dental insurance | Disability insurance | Health insuranceSenior-level Full TimeEnglewood Cliffs, NJ; Santa Clara, CA15h ago