Software Engineer — Distributed LLM Inference Systems
Tasks
- Debug and optimize software performance
- Design distributed LLM inference systems
- Develop request schedulers and model workers
- Implement distributed inference algorithms
- Improve end to end latency throughput scalability resource utilization
- Manage KV cache mechanisms
- Optimize communication between inference components
- Optimize model execution performance
- Profile inference workloads
- Write code, tests, and documentation
Perks/Benefits
Skills/Tech-stack
Asynchronous Communication | C plus plus | Continuous batching | Data parallelism | Deep learning | Distributed Systems | KV cache | Language Models | Large Language Models | Machine Learning | Model Parallelism | Open Source | Open-source collaboration | Performance optimization | Prefill and Decode | Profiling | PyTorch | Python | Request Scheduling | SGLang | TensorRT-LLM | VLLM
Education
Roles
Related jobs
-
Artificial Intelligence | CIP | Cause analysis | Collaborative robots | Control SystemsSenior-level Full TimeWuxi, Jiangsu, China11h ago
-
Mid-level Full Time北京19h ago
-
None Full Time上海19h ago
-
Entry-level Internship上海20h ago
-
AI software application developers (CG) CNY 84K-120KAI | Algorithms | Benchmarking | C# | C++Career growth | Continuous learning | Onsite workEntry-level Full TimeCHN - Minhang, China1d ago
-
AI Framework Software Engineer - vLLM CNY 300K-420KAsynchronous Communication | C++ | Data parallelism | Deep learning | Distributed SystemsEntry-level Full TimeCHN - Minhang, China1d ago
-
Mid-level Full TimeXian, Shaanxi, China1d ago
-
Mid-level Full TimeShanghai, China1d ago
-
AI Frameworks Engineer - Model and Kernel Optimization CNY 160K-240KC++ | Deep learning | GPU Programming | Kernel development | PyTorchOn-site workSenior-level Full TimeCHN - Minhang, China1d ago
-
AI Framework Software Engineer – PyTorch CNY 74K-100KC++ | Debugging | Deep learning | Distributed Computing | GPU OptimizationOn-site workEntry-level Full TimeCHN - Minhang, China1d ago
-
AI Compiler and Library Engineer CNY 60K-60KAssembly | C# | C++ | Deep learning | JavaCollaborative environment | Hands-on projects | On-site workEntry-level Full TimeCHN - Minhang, China1d ago
-
Data Analyst Engineer A1 CNY 360K-510KAgile | Bash | CI/CD | CSS | CloudEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China1d ago
-
Senior-level Full TimeCHN - ZJ - Hangzhou, Innovation …1d ago
-
Mid-level Full Time北京1d ago
-
Miclaw-端云协同调度专家 (Hybrid AI Architect) CNY 144K-240KClaude 3.5 | Consistency protocols | Context Switching | Data Compression | Distributed SystemsHybrid workSenior-level Full Time北京 R1d ago
-
AI基础设施研发工程师（Sandbox / 容器化)-MiMo CNY 144K-240KAlerting | AppArmor | Argo Workflows | CI/CD | CPU SchedulingMid-level Full Time北京1d ago
-
Senior-level Full Time北京1d ago
-
Algorithms | Artificial Intelligence | Concurrency | Containerization | Data StructuresMid-level Full TimeShanghai2d ago
-
Apache Flink | Apache Kafka | Apache Spark | Apache Superset | Data GovernanceMid-level Full TimeShanghai2d ago
-
Machine Learning Engineer CNY 180K-360KAndroid | C# | C++ | Convolutional Neural Network | Deep learningEntry-level Full TimeShanghai, China2d ago
-
Senior-level Full TimeShanghai, Shanghai, China2d ago
-
Senior-level Full TimeShanghai Jing'An Office, China2d ago
-
None Full Time上海2d ago
-
Entry-level Full Time北京2d ago
-
机器人VLA算法研究员 - XiaomiRobotics CNY 500K-500KAction Generation | Data Modeling | Deep learning | Diffusion Models | Machine LearningMid-level Full Time北京2d ago