Software Engineer, Inference Platform
Tasks
- Build and operate KV cache and scheduling infrastructure
- Contribute to inference platform architecture and roadmap
- Drive improvements in throughput TTFT and cost per token
- Implement and validate disaggregated prefill and decode pipelines
- Own inference deployments end-to-end
- Participate in on-call rotation to maintain system reliability
- Partner with customers to optimize deployment configurations
- Profile and resolve bottlenecks across compute memory and communication
Perks/Benefits
Skills/Tech-stack
CUDA | Distributed Systems | Expert parallelism | GPU Compute | GPU Optimization | GPU compute parallelism | GPU memory | GPU memory hierarchies | Go | Inference Engines | JAX | Kubernetes | LLM serving | LLM serving frameworks | Memory hierarchies | Model Deployment | PyTorch | Python | Quantization tooling | Serving frameworks | Speculative decoding | Tensor and expert parallelism | Torch.compile | Triton
Education
Roles
Regions
Countries
States
Related jobs
-
Automation Testing | CI/CD | CSS | Cypress | Feature DevelopmentMedical, dental & vision coverage | Paid time off | Parental leave | Reimbursement programs | Retirement planMid-levelRaleigh, United States R11d ago
-
Robotics Test & Data Engineer, Mapping USD 70K-300K3D Point Cloud Processing | 3D point cloud | Bash | Cloud processing | Cloud validationMid-level Full TimeIrvine, CA8h ago
-
Senior-level Full TimeSan Francisco, US9h ago
-
Senior-level Full TimeUSA - Atlanta, GA; USA - …10h ago
-
Senior Software Engineer - Data Fulfillment USD 155K-190KAPI | AWS | Azure | C# | CI/CD401k plan | Dental insurance | Flexible paid time off | Flexible work hours | Medical insuranceSenior-level Full TimeSeattle, WA12h ago
-
Staff Software Engineer - Distributed Data Systems USD 182K-247KACID transactions | Algorithms | Apache Spark | C++ | Data StructuresSenior-level Full TimeBellevue, Washington12h ago
-
Senior Software Engineer - Distributed Data Systems USD 157K-213KACID transactions | Algorithms | Apache Spark | C++ | Data StructuresSenior-level Full TimeBellevue, Washington12h ago
-
Partner 20, Applied ML, Engineer, ASG USD 362K-422KAirflow | CI/CD | Data Engineering | Docker | Feature EngineeringMid-level Full TimeSan Francisco, California, United States12h ago
-
Senior AI Software Engineer USD 160K-198KA2A | ADK | API | Containers | EnvoyNo visa sponsorship | Remote workSenior-level Full TimeRemote US R13h ago
-
Deployed Engineer (Seattle) USD 150K-250KAWS | Agent Orchestration | Containers | Demo Delivery | Failure handling401k plan | Dental insurance | Flexible vacation | Health insurance | Life insuranceMid-level Full TimeSeattle, WA14h ago
-
Data Scientist, Underwriting USD 190K-300KA/B | A/B Testing | B testing | Causal Inference | Data Analysis401k | Commuter benefits | Employee assistance program | Free Lunches | Medical, dental, and vision insuranceMid-level Full TimeSan Francisco, CA14h ago
-
AI & ML Engineering Intern USD 104K-166KAutomation | DBT | Data Connectors | Embeddings | EvaluationEquity | Flexible schedule | Free snacks | Paid time off | Ping-pong tableEntry-level InternshipAustin, Texas14h ago
-
Senior Machine Learning Engineer USD 120K-195KAnomaly Detection | Apache Beam | Apache Spark | BigQuery | CI/CD401k | Compassionate leave | Dental insurance | Discount Portal | HolidaysSenior-level Full TimeRemote - US R14h ago
-
Systems Software Engineer - Robotic Platform USD 150K-200K*nix | C++ | Deterministic builds | Docker | LinuxSenior-level Full TimeAustin, Texas, United States14h ago
-
Senior Machine Learning Engineer USD 218K-273KA/B | A/B Testing | B testing | CatBoost | Embeddings401k | Employee assistance program | Equity compensation | Flexible PTO | HSA/FSASenior-level Full TimeNew York, New York, United States; …14h ago
-
Software Engineer - Robotic Applications USD 160K-245KAutonomy Models | Behavior systems | C++ | Controls | LocalizationSenior-level Full TimeAustin, Texas, United States14h ago
-
Staff Machine Learning SWE Infra USD 238K-302KAutoregressive models | Cloud Computing | Data Engineering | Distributed Training | Gradient ShardingCompany benefits program | Discretionary annual bonus | Equity incentive planSenior-level Full TimeMountain View, CA, USA15h ago
-
Mid-level Full TimeBoston, MA15h ago
-
Senior-level Full TimeBoston, MA15h ago
-
Senior Developer, Data & IT - AI Solutions USD 120K-142KAI Agents | API Integration | AWS | AWS Bedrock | AWS SageMakerDental insurance | Dependent Care Account | Health insurance | Health savings account | Mental health counseling supportSenior-level Full TimeNew York, NY, United States15h ago
-
Member of Technical Staff - Hardcore Supercompute USD 180K-440KAnsible | C plus plus | Containerization | Debugging | Distributed Systems401k retirement plan | Dental insurance | Disability insurance | Employee discounts | Life insuranceSenior-level Full TimePalo Alto, CA; Seattle, WA15h ago
-
Member of Technical Staff - Infrastructure Supercompute USD 180K-370KAnsible | ArgoCD | Docker | Flux | Go401k retirement plan | Dental insurance | Discounts and perks | Life insurance | Long-term disability insuranceSenior-level Full TimePalo Alto, CA; Seattle, WA15h ago
-
AI and Automation Engineer USD 135K-250KAirbyte | Alerting | Amazon Web Services | Application Programming | Application Programming InterfacesCareer growth opportunities | Company-Paid Holidays | Medical/Dental/Vision insurance | Paid time offMid-level Full TimeAustin, TX15h ago
-
AI Engineer (Hybrid @ Bellevue, WA or Remote @ Florida) USD 140K-160KAWS | Artificial Intelligence | Asyncio | Evals | Function Calling401k match | Disability benefits | Family leave | Health insurance | Life insuranceEntry-level Full TimeHybrid @ Bellevue, WA or Remote … R15h ago
-
Senior Data Engineer USD 120K-142KApache Airflow | C# | Continuous integration | DBT | Data Modeling401b retirement savings program | Dental insurance | Dependent Care Account | Employer match | Flexible spending accountsSenior-level Full TimeNew York, NY, United States15h ago