Senior Software Engineer - AI Inference
Tasks
- Author pull requests and participate in code reviews
- Build and maintain performance and correctness regression tests
- Collaborate with model platform and SRE teams to deliver upstream solutions
- Contribute features fixes optimizations upstream vLLM and SGLang
- Implement optimize inference runtime capabilities batching scheduling streaming request lifecycle management KV cache paging and sharding
- Improve multi GPU inference performance and reliability
- Profile and optimize performance bottlenecks across Python and C plus plus CUDA kernels
Perks/Benefits
- N/A
Skills/Tech-stack
Attention | Batching | C++ | CUDA | Concurrency | Distributed Systems | Flame Graphs | GPU Profiling | Infiniband | KV cache | Microbenchmarking | NCCL | NVLink | PCIe | Paging | Parallelism | PyTorch | Python | Quantization | SGLang | Scheduling | Sharding | Speculative decoding | Streaming | Triton | VLLM
Education
Regions
Countries
States
Related jobs
-
C plus plus | CUDA | Computer Graphics | Computer Vision | Deep learningComprehensive health benefits | Equity | Family benefits | Relocation assistanceSenior-level Full TimeUS, CA, Santa Clara R2d ago
-
Senior Software Engineer, AI Performance Analysis USD 184K-356KAI Inference | AI Training | AI workloads | C++ | Developer toolsBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R2d ago
-
Software Engineer - Drive OS USD 124K-195KAgent Orchestration | Bazel | C++ | CI/CD | CMakeEquity | Health benefits | Paid time offSenior-level Full TimeUS, CA, Santa Clara R3d ago
-
Senior Platform AI Engineer USD 184K-356KAuthentication | Authorization | C# | C++ | CachingComprehensive benefitsSenior-level Full TimeUS, CA, Santa Clara R4d ago
-
Applied Agentic AI Lead, Partner Co-Design USD 224K-431KAgent systems | Compliance | Deep learning | Docker | Fine TuningBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R4d ago
-
C# | C++ | CPU architecture | CUDA | GPU ArchitectureComprehensive benefits package | Equity | Mentoring opportunitiesSenior-level Full TimeUS, CA, Santa Clara R4d ago
-
Senior-level Full TimeUS, CA, Santa Clara R5d ago
-
C++ | Caching | Debugging | Distributed Systems | GPU memorySenior-level Full TimeUS, CA, Santa Clara R5d ago
-
Benchmarking | C++ | Code generation | Debugging | GPU ComputingSenior-level Full TimeUS, CA, Santa Clara R5d ago
-
Senior Solutions Architect, Embedded Physical AI USD 152K-287KCompression | Computer Vision | CuVSLAM | Edge AI | Embedded SystemsSenior-level Full TimeUS, CA, Santa Clara R11d ago
-
Senior Storage Software Engineer, DGXC Data Services USD 152K-287KAPI Design | Algorithms | C++ | Caching | ConcurrencySenior-level Full TimeUS, CA, Santa Clara R12d ago
-
Senior Software Engineer - AI Research Clusters USD 152K-287KAIOps | Agentic AI | C++ | CSS | Database ModelingEquity | Health insurance | Paid time off | Retirement planSenior-level Full TimeUS, CA, Santa Clara R12d ago
-
C++ | Cache Management | Distributed Systems | GPU memory | GPU memory managementAgile team environment | Equity | Health benefitsSenior-level Full TimeUS, CA, Santa Clara R16d ago
-
Senior Software Engineer, DGXC Data Services USD 152K-287KAWS | Algorithms | Apache Iceberg | Apache Spark | AzureEmployee benefits | EquitySenior-level Full TimeUS, CA, Santa Clara R23d ago
-
Bash | Bootstrap | CSI | CSS3 | Container StorageSenior-level Full TimeUS, CA, Santa Clara R26d ago
-
Senior Software Engineer, AI Storage USD 184K-287KAlgorithms | Bash | C++ | CUDA | CloudBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R26d ago
-
Senior Deep Learning Framework Communications Engineer USD 152K-287KC++ | CUDA | CUDA kernels | CuTe | Distributed TrainingBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R28d ago
-
C++ | CUDA | Docker | Infiniband | JAXSenior-level Full TimeUS, CA, Santa Clara R29d ago
-
Senior Deep Learning Frameworks CUDA Software Engineer USD 184K-356KAutograd | C++ | CUDA | Compiler technology | Computer ArchitectureSenior-level Full TimeUS, CA, Santa Clara R1mo ago
-
Senior Scientific Machine Learning Engineer – Earth-2 USD 152K-287KCUDA | Containers | Data parallelism | Diffusion Models | GPU KernelBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R1mo ago
-
Senior Storage Production Engineer - DGX Cloud USD 176K-333KAI/ML | Access Control | Algorithms | Ansible | AuditingBenefits | Equity | On-call rotationSenior-level Full TimeUS, CA, Santa Clara R1mo ago