Senior System Software Engineer, Agentic Inference - Dynamo
Tasks
- Build distributed inference frontend
- Contribute to disaggregated inference serving
- Deliver day zero support for new models
- Develop open source GPU inference software
- Implement stateful Responses API semantics
- Innovate inference state management
- Integrate latest open source technology
- Load-balance asynchronous inference requests
- Optimize throughput under latency constraints
- Support agentic inference workloads
Perks/Benefits
- N/A
Skills/Tech-stack
Concurrency | Context Management | Debugging | Distributed Systems | GPU Memory Optimization | GPU memory | High Performance | High-performance networking | KV cache | LLM API | LLM API Semantics | Memory Optimization | Multimodal inputs | Performance Analysis | Prefix Cache | Python | Rust | Scheduling | Software Design | Structured Output | Testing | Token Accounting | Tool-Calling
Education
Regions
Countries
States
Related jobs
-
Senior Agentic AI For Data Science Applied Researcher USD 152K-287KAblation Studies | Agentic AI | Benchmarking | Coding Agents | Data CurationComprehensive benefits package | EquitySenior-level Full TimeUS, CA, Santa Clara R1d ago
-
Senior Machine Learning Engineer USD 184K-287KAWS | Distributed Systems | Kubernetes | Machine Learning | Optimization AlgorithmsEmployee benefits | EquitySenior-level Full TimeUS, CA, Santa Clara R1d ago
-
Senior Software Engineer, CUDA Deep Learning Systems USD 184K-356KC++ | CUDA | CUDA kernels | Computer Architecture | Deep learningBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R1d ago
-
Senior Deep Learning Frameworks CUDA Software Engineer USD 184K-356KC++ | CUDA | Distributed machine learning | HPC | JAXSenior-level Full TimeUS, CA, Santa Clara R1d ago
-
Senior Cloud Software Engineer, DGXC Data Services USD 152K-241KAPI Design | AWS | Algorithms | Azure | C++Senior-level Full TimeUS, CA, Santa Clara R2d ago
-
Senior Software Engineer, AI Storage USD 184K-287KAlgorithms | Bash | C plus plus | C# | CUDAEquity | Health insuranceSenior-level Full TimeUS, CA, Santa Clara R2d ago
-
Senior Deep Learning Infrastructure Engineer USD 184K-356KApplication Programming | Application Programming Interface | Cloud Computing | Data pipeline | DatabaseSenior-level Full TimeUS, CA, Santa Clara R4d ago
-
C++ | CUDA | Containers | Distributed Training | Distributed inferenceEquity | Health benefits | Paid time offSenior-level Full TimeUS, CA, Santa Clara R4d ago
-
Architecture Search | CUDA | Cutlass | Huggingface | JAXSenior-level Full TimeUS, CA, Santa Clara R6d ago
-
Senior Software Engineer – TensorRT Edge-LLM USD 152K-241KAttention | Autoregressive Model | Autoregressive model serving | Benchmarking | C++Senior-level Full TimeUS, CA, Santa Clara R6d ago
-
Bash | Bootstrap | CSI | CSS3 | Container StorageSenior-level Full TimeUS, CA, Santa Clara R16d ago