MaaS 架构师
Tasks
- Apply prefix caching and paged attention
- Build inference gateway with model routing and traffic scheduling
- Create unified inference load abstraction and service wrapper
- Deploy quantized models
- Design GPU autoscaling and resource scheduling
- Design end to end model service architecture
- Enable token level metering
- Ensure production SLA latency and throughput
- Implement KV cache aware scheduling
- Implement observability for model services
- Implement priority queue and multi tenant isolation
- Optimize inference engines with continuous batching and speculative decoding
- Select and tune single node inference framework
Perks/Benefits
- N/A
Skills/Tech-stack
Attention | Batching | C++ | CUDA | Cluster architecture | Continuous batching | GPU Cluster | GPU Cluster Architecture | GPU Programming | GPU autoscaling | GPU communication | Go | KV cache | Kubernetes | LLM | Long Context | Long context inference | MOE | Model routing | Multi-model | Multi-model routing | Multi-tenancy | NCCL | Paged Attention | Prefix caching | Priority queue | Python | Quantization | Resource scheduling | SGLang | Speculative decoding | TensorRT-LLM | Token metering | Traffic scheduling | Triton | VLLM
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Regions
Countries
States
Related jobs
-
Senior-level Full Time上海、武汉、北京1h ago
-
Mid-level Full Time北京、上海2h ago
-
Senior-level Full Time北京、上海2h ago
-
AI Data 研发工程师（大模型数据方向） CNY 180K-264KAirflow | Airflow Workflows | Arrow | Caption Generation | ClusteringMid-level Full Time上海、北京2h ago
-
AGI 研发 Leader（AI Agent / AI App） CNY 144K-240KBackend Development | Benchmarking | Data Processing | Dify | Distributed SystemsSenior-level Full Time北京、上海2h ago
-
AGI 服务端资深工程师-Talkie&星野 CNY 144K-240KBackend Development | Dify | Inference Optimization | LLM | LangchainSenior-level Full Time北京、上海2h ago
-
AI Engineer | Financial Services HKD 425K-586KAWS | Agent-based | Agent-based systems | Azure | CCPAMid-level Full TimeChina - Hong Kong14h ago
-
Deep Learning Solution Architect - RL and Post-training CNY 360K-600KAgentic AI | CUDA | Cloud Architecture | DPO | Distributed TrainingSenior-level Full TimeChina, Beijing22h ago
-
Senior-level Full Time武汉、北京3d ago
-
Ai全栈工程师 CNY 144K-240KAmazon Web Services | CI/CD | Docker | Express | FastAPIAI tools license | Flexible work hours | Remote work optionSenior-level Full Time北京、南京、苏州4d ago
-
AWS | Access Management | Agent systems | Agentic Workflows | AuditabilityMid-level Full TimeShenzhen, Guangdong, China R4d ago
-
AWS | Agent Orchestration | Agent systems | Azure | Cloud platformMid-level Full TimeShenzhen, Guangdong, China R4d ago
-
APIs | AWS | Accuracy Optimization | Agent Orchestration | Agent systemsMid-level Full TimeShenzhen, GD, CN, 5180104d ago
-
Data platform architect CNY 3000KAccess Control | Apache Spark | Auto Loader | Azure Data | Azure Data FactorySenior-level Full TimeCN, Shanghai, China4d ago
-
Deep Learning Solution Architect - Agentic Performance CNY 360K-600KAI Agents | Agentic Inference | Cluster management | Distributed Training | Distributed inferenceSenior-level Full TimeChina, Beijing4d ago
-
Senior Deep Learning Solution Architect CNY 160K-240KAccelerated computing | C plus plus | Computer Architecture | Data Structures | Distributed TrainingSenior-level Full TimeChina, Beijing4d ago
-
Senior Deep Learning Solution Architect CNY 160K-240KAccelerated computing | C++ | CPU | Computer Architecture | Computer SystemsSenior-level Full TimeChina, Beijing4d ago
-
Solutions Architect - Financial Services CNY 360K-500KAgentic AI | Algorithm Trading | C# | C++ | CUDASenior-level Full TimeChina, Beijing4d ago
-
Senior-level Full TimeChina, Beijing4d ago
-
Mid-level Full TimeShanghai, Shanghai, China4d ago
-
AI Infra工程师-算法研发方向 CNY 180K-300KC plus plus | CI/CD | CUDA | Data parallelism | Deep learningEngineering Training | MentorshipMid-level Full Time北京、上海5d ago
-
Entry-level Full Time InternshipChangsha, Shenzhen5d ago
-
Ai全栈工程师（25/26届） CNY 96K-120KAI integration | API Development | Full Stack | Full-Stack Development | GoLearning opportunities | MentorshipEntry-level Full Time InternshipShenzhen, Changsha5d ago
-
API Integration | Agent Design | Agent evaluation | Context engineering | Data EngineeringEntry-level Full Time InternshipShenzhen5d ago
-
Executive-level Full TimeHangzhou, China5d ago