Senior Engineer, Inference Data Plane
Tasks
- Architect and refine system design for multi tenant AI inference cloud
- Build Kubernetes native distributed inference frameworks
- Coach and mentor junior engineers
- Contribute upstream to llm d vLLM and inference gateway projects
- Drive end to end design development and delivery of data plane services
- Implement and optimize distributed inference hosting
- Implement prefill decode disaggregation and KV cache aware routing
- Maintain and operate high scale services with observability and SLOs
- Solve inference aware load balancing and flow control across tenants
Perks/Benefits
- Employee assistance program
- Flexible time off
- Hybrid work model
- LinkedIn Learning access
- Local Employee Meetups
- Reimbursement for conferences training and education
Skills/Tech-stack
Autoscaling | Continuous batching | Data parallelism | GRPC | Go | KServe | KV cache | Kubernetes | Llm-d | NVIDIA TensorRT | Paged Attention | Prefix caching | Python | Quantization | Ray Serve | SGLang | Tensor Parallelism | TensorRT-LLM | VLLM
Education
N/A
Related jobs
-
Mid-Level Data Engineer USD 130K-140KAPI first | API-first design | AWS | Agile | Amazon RedshiftRemote workMid-level Full TimeWork from home, VA, United States2h ago
-
Sr. Data Engineer USD 150K-160KAWS | Agile | Automated testing | Big Data | Continuous integrationRemote workSenior-level Full TimeWork from home, VA, United States2h ago
-
Data Engineer IV USD 110K-155KAWS | Azure | Azure Data | Azure Data Factory | CI/CD401k | Accident insurance | Continuing education assistance | Critical illness insurance | Employee assistance programSenior-level Full TimeNorthern Virginia, VA; Northern Virginia, VA; …3h ago
-
Data Engineer III USD 95K-130KAWS | Azure | Azure Data | Azure Data Factory | CI/CD401k | Accident insurance | Continuing education assistance | Critical illness insurance | Employee assistance programSenior-level Full TimeNorthern Virginia, VA; Northern Virginia, VA3h ago
-
Data Engineer II USD 75K-105KData Pipelines | Data Quality | Data Validation | Data Warehousing | Data integration401k | Accident insurance | Continuing education support | Critical illness insurance | Employee assistance programEntry-level Full TimeNorthern Virginia, VA; Northern Virginia, VA; …3h ago
-
Azure Data | Azure Data Factory | Azure DevOps | Azure Machine Learning | CI/CDRemote | USC OnlySenior-level Full TimeDallas, TX. R3h ago
-
Data Scientist / Machine Learning Engineer USD 110K-150KAPIs | Agile | CI/CD | Computer Vision | Continuous Deployment401k | Dental insurance | Flexible spending account | Health insurance | Life and AD&D insuranceMid-level Full TimeSan Diego, California; San Diego, CA; …3h ago
-
ABAC | AWS CloudFormation | AWS Lambda | AWS Step Functions | AWS-APIsOnsite/hybrid workSenior-level Full TimeChantilly, Virginia R4h ago
-
AI Agents | AI Workflow Orchestration | AI workflow | AWS | AWS LambdaDental | Medical | Paid time off (PTO) | Remote work | RetirementSenior-level Full TimeLos Angeles, CA, United States4h ago
-
AWS Lambda | AWS Step Functions | Amazon Web Services | Anthropic | Cloud IntegrationsDental insurance | Medical insurance | Paid time off | Remote work | Retirement benefitsMid-level Full TimeIrvine, CA, United States4h ago
-
AUTOSAR | C++ | CAN | DaVinci Developer | Hardware-in-the-loopSenior-level Full TimeDearborn, United States5h ago
-
AI Developer Builder USD 140K-180KAWS | Agentic Systems | Anthropic Bedrock | Context pipelines | DatadogDirect input on roadmap | High ownershipSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL R6h ago
-
Artificial Intelligence Machine Learning Engineer USD 98K-163KArtificial Intelligence | CI/CD | Docker | Elasticsearch | FastAPIHealth insurance | Holiday pay | Learning and development | Life insurance | Long-term disabilitySenior-level Full TimeUSA-VA-Ashburn7h ago
-
AI Software Engineer USD 260K-260KAI Search | API Integration | Agent Orchestration | Azure AI | Azure AI SearchCareer growth | Healthcare | On-site childcare | PTO | Paid HolidaysSenior-level Full TimeTexas-Dallas-5323 Harry Hines Blvd7h ago
-
Compression | Data Engineering | Data Pipelines | Distributed Systems | High AvailabilitySenior-level Full TimeSan Jose, California, United States7h ago
-
Senior Production Engineer - Applied Machine Learning USD 218K-480KAlerting | Auto Scaling | Auto rollback | Auto-healing | CI/CDSenior-level Full TimeSan Jose, California, United States7h ago
-
Data Scientist Graduate (Multimedia) - 2027 Start USD 128K-177KA/B | A/B Testing | B testing | Causal Inference | Data MiningEntry-level Full TimeSan Jose, California, United States7h ago
-
DPO | DeepSpeed | Distributed Training | Evaluation Pipelines | Human preference learningEntry-level Full TimeSan Jose, California, United States7h ago
-
Senior Data Engineer (Multiple Positions) USD 220K-387KApache Flink | Apache Spark | Architecture Design | Data Governance | Data ModelingSenior-level Full TimeSan Jose, California, United States7h ago
-
C++ | Clustering | Computer Vision | Data Mining | DeduplicationGrowth opportunities | Team collaborationEntry-level Full TimeSeattle, Washington, United States7h ago
-
C++ | Clustering | Computer Vision | Content deduplication | Data EngineeringEntry-level Full TimeSan Jose, California, United States7h ago
-
Adversarial Robustness | Apache Hadoop | Apache Hive | Apache Spark | C++Senior-level Full TimeSan Jose, California, United States7h ago
-
Gen AI Developer Specialist USD 105K-204KAI Evaluation | AWS | Access Management | Agents | Amazon BedrockSenior-level Full TimeUnited States8h ago
-
Software Engineer, MTIA SW Tooling Engineer USD 141K-208KC++ | CUDA | CUDA GDB | Cuda Memcheck | DWARFSenior-level Full TimeBellevue, WA | Menlo Park, CA …8h ago
-
AI Hardware Accelerator | AI Practices | AI hardware | ASICs | Allocation OptimizationSenior-level Full TimeMenlo Park, CA8h ago