Senior Software Engineer – TensorRT Edge-LLM
Tasks
- Benchmark and profile inference performance
- Collaborate with CUDA kernel and compiler teams
- Design compiler optimizations for transformer models
- Design runtime optimizations for transformer models
- Develop CUDA kernels and operators
- Develop inference framework in modern C++
- Extend TensorRT with autoregressive model serving
- Implement LoRA
- Implement mixture of experts
- Implement speculative decoding
- Integrate emerging LLM techniques into product software
- Manage KV cache
- Optimize GEMM performance
- Optimize MoE performance
- Optimize attention performance
- Optimize inference performance for embedded and automotive
Perks/Benefits
- N/A
Skills/Tech-stack
Attention | Autoregressive Model | Autoregressive model serving | Benchmarking | C++ | CUDA | CUDA kernels | Cache Management | Compiler optimizations | FlashInfer | GEMM | KV cache | KV-cache management | LoRA | MLC LLM | Memory-efficient scheduling | Mixture of Experts | Model Serving | Profiling | Quantization | Runtime Optimizations | SGLang | Speculative decoding | Tensor Parallelism | TensorRT | Transformer Models | VLLM
Education
Regions
Countries
States
Related jobs
-
Senior Agentic AI For Data Science Applied Researcher USD 152K-287KAblation Studies | Agentic AI | Benchmarking | Coding Agents | Data CurationComprehensive benefits package | EquitySenior-level Full TimeUS, CA, Santa Clara R1d ago
-
Senior Software Engineer, CUDA Deep Learning Systems USD 184K-356KC++ | CUDA | CUDA kernels | Computer Architecture | Deep learningBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R1d ago
-
Senior Deep Learning Frameworks CUDA Software Engineer USD 184K-356KC++ | CUDA | Distributed machine learning | HPC | JAXSenior-level Full TimeUS, CA, Santa Clara R1d ago
-
Senior Cloud Software Engineer, DGXC Data Services USD 152K-241KAPI Design | AWS | Algorithms | Azure | C++Senior-level Full TimeUS, CA, Santa Clara R2d ago
-
Senior Software Engineer, AI Storage USD 184K-287KAlgorithms | Bash | C plus plus | C# | CUDAEquity | Health insuranceSenior-level Full TimeUS, CA, Santa Clara R2d ago
-
C++ | CUDA | Containers | Distributed Training | Distributed inferenceEquity | Health benefits | Paid time offSenior-level Full TimeUS, CA, Santa Clara R4d ago
-
Architecture Search | CUDA | Cutlass | Huggingface | JAXSenior-level Full TimeUS, CA, Santa Clara R6d ago
-
Concurrency | Context Management | Debugging | Distributed Systems | GPU Memory OptimizationSenior-level Full TimeUS, CA, Santa Clara R8d ago