大模型推理与部署优化工程师
Tasks
- Adapt models to NVIDIA and Ascend accelerators
- Build and optimize inference engine
- Containerize and orchestrate model services with Docker and Kubernetes
- Deploy multimodal model online
- Design and implement high performance model serving architecture
- Diagnose and resolve online bottlenecks including OOM and tail latency
- Handle load balancing and streaming output
- Implement prefill decode separation and continuous batching
- Implement speculative decoding and prefix caching
- Monitor services and troubleshoot crashes and performance issues
- Optimize operators for hardware compatibility and performance
- Perform autoscaling and operational stability support
- Standardize and engineering model weights with algorithm teams
- Tune inference performance for high concurrency low latency
Perks/Benefits
- N/A
Skills/Tech-stack
Ascend 910B | Ascend CANN | Autoscaling | CUDA | Continuous batching | Docker | KV cache | Kubernetes | LightLLM | Linux | Load Balancing | Monitoring and Alerting | NVIDIA A100 | NVIDIA H100 | Nsight Systems | Observability | PagedAttention | Prefix caching | PyTorch | PyTorch Profiler | SGLang | Speculative decoding | Streaming output | TensorRT-LLM | VLLM
Education
Related jobs
-
Senior AI Operations Engineer CNY 144K-240KAlert Correlation | Docker | Forecasting | Function Calling | KubernetesSenior-level Full TimeChina, China2d ago
-
Consultant Specialist CNY 144K-184KAlibaba Cloud | CI/CD | Cloud deployment | Cloud platform | ContainerizationSenior-level Full TimeXi'an, Shaanxi, China2d ago
-
Senior Consultant Specialist（GenAI Agent platform） CNY 160K-240KAWS | Access Management | Agent Orchestration | Audit Logging | AzureSenior-level Full TimeGuangzhou, Guangdong, China R2d ago
-
Consultant Specialist CNY 144K-184KAlibaba Cloud | CI/CD | Containerization | Cost Optimization | Data GovernanceSenior-level Full TimeXi'an, SN, CN, -2d ago
-
Mid-level Full TimeBJN01 - DXC Beijing LSH Center …2d ago
-
Mid-level Full TimeShanghai Jing'An Office, China2d ago
-
Mid-level Full TimeChina Shanghai2d ago
-
Mid-level Full TimeChina Shanghai2d ago
-
腾讯云数据迁移工程师 CNY 144K-240KAWS | Azure | CI/CD | CloudFormation | Data GovernanceFlexible work arrangements | In-person collaborationMid-level Full TimeSHC01 - DXC Shanghai Campus Phase …2d ago
-
数据平台开发工程师 CNY 180K-300KData Lake | Data Warehouse | Data Warehouse Modeling | Data pipeline | DataOpsMid-level Full Time广州2d ago
-
Mid-level Full TimeShanghai, China3d ago
-
Senior Machine Learning Engineer CNY 400K-510K3D Reconstruction | Adversarial Networks | C++ | Computer Graphics | Computer VisionSenior-level Full TimeShanghai, China3d ago
-
Mid-level Full TimeCN-OCG International Center, Cheng Du, China3d ago
-
Senior-level Full Time深圳3d ago
-
Agile | Apache Spark | Continuous integration | Data Pipelines | Data QualityMid-level Full TimeShanghai4d ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous work culture | Friendly work environment | Hands-off management | Opportunity to make impactMid-level Full TimeBeijing, China4d ago
-
Machine Learning Engineer CNY 144K-240KData Curation | Data Quality | Deep learning | Fine Tuning | JAXSenior-level Full TimeBeijing4d ago
-
AWS | Apache Airflow | Apache Doris | Apache Iceberg | Apache SparkSenior-level Full TimeShanghai4d ago
-
Entry-level Full Time上海 R4d ago
-
多模态交互算法工程师-X-Lab CNY 240K-480KAttention | Checkpointing | Hard Negative Mining | KV cache | Knowledge DistillationSenior-level Full Time上海、深圳4d ago
-
Capacity Planning | Cloud services | Cost modeling | Data Analysis | Machine LearningSenior-level Full TimeShanghai5d ago
-
Senior Machine Learning Engineer CNY 144K-240KData Preparation | GPU Inference | GPU Training | Inference Optimization | JAXSenior-level Full TimeChina5d ago
-
Staff Machine Learning Engineer CNY 144K-240KBias detection | DPO | Data Pipelines | Evaluation Pipelines | Fine TuningSenior-level Full TimeChina5d ago
-
ASIC | Agent systems | Agentic AI | BF16 | Bandwidth OptimizationSenior-level Full TimeShanghai, China5d ago
-
R&D – IoT Robotics Engineer CNY 360K-540KC++ | CI/CD | Computer Vision | Control Systems | Domain RandomizationSenior-level Full TimeGuangdong, China5d ago