大模型推理与部署优化工程师
Tasks
- Adapt models to NVIDIA and Ascend accelerators
- Build and optimize inference engine
- Containerize and orchestrate model services with Docker and Kubernetes
- Deploy multimodal model online
- Design and implement high performance model serving architecture
- Diagnose and resolve online bottlenecks including OOM and tail latency
- Handle load balancing and streaming output
- Implement prefill decode separation and continuous batching
- Implement speculative decoding and prefix caching
- Monitor services and troubleshoot crashes and performance issues
- Optimize operators for hardware compatibility and performance
- Perform autoscaling and operational stability support
- Standardize and engineering model weights with algorithm teams
- Tune inference performance for high concurrency low latency
Perks/Benefits
- N/A
Skills/Tech-stack
Ascend 910B | Ascend CANN | Autoscaling | CUDA | Continuous batching | Docker | KV cache | Kubernetes | LightLLM | Linux | Load Balancing | Monitoring and Alerting | NVIDIA A100 | NVIDIA H100 | Nsight Systems | Observability | PagedAttention | Prefix caching | PyTorch | PyTorch Profiler | SGLang | Speculative decoding | Streaming output | TensorRT-LLM | VLLM
Education
Related jobs
-
Mid-level Full Time北京18h ago
-
Mid-level Full Time北京18h ago
-
多模态数据算法工程师 CNY 216K-360KComputer Vision | Data Mining | Data Observability | Data cleaning | Deep learningEntry-level Full Time上海18h ago
-
具身智能数据算法工程师 CNY 144K-240K3D Gaussian Splatting | 3D Vision | Amazon S3 | Anomalous Data Detection | Apache ArrowMid-level Full Time北京18h ago
-
Mid-level Full Time深圳18h ago
-
多模态大模型算法工程师 CNY 240K-480KAblation Study | Agile | Computer Vision | Data Annotation | Data DeduplicationSenior-level Full Time北京18h ago
-
多模态大模型算法工程师 CNY 240K-480KComputer Vision | Data Augmentation | Data Deduplication | Data Mining | Data PreprocessingSenior-level Full Time深圳18h ago
-
多模态大模型算法工程师 CNY 240K-480KAgile | Computer Vision | Data Annotation | Data Deduplication | Data FilteringSenior-level Full Time上海18h ago
-
多模态大模型算法工程师 CNY 240K-480KAgile | Attribution Analysis | Automatic Labeling | Computer Vision | Data AblationSenior-level Full Time成都18h ago
-
Data Platform Engineering-Senior Principal, VP CNY 360K-600KAI Agent | AI Agent Development | AWS | Agent Development | Agent systemsEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China19h ago
-
Data Engineering - AI Platform Engineer CNY 300K-380KAWS | Access Control | Authentication | Azure | CI/CDEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysMid-level Full TimeHangzhou, China19h ago
-
Entry-level Internship Part TimeWuxi, Jiangsu, China19h ago
-
Data Engineering - Agentic Engineer CNY 360K-540KAWS | Access Control | Apache Iceberg | Apache Spark | CI/CDEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer days | Vibrant employee networksSenior-level Full TimeHangzhou, China19h ago
-
Mid-level Full Time北京1d ago
-
Mid-level Full Time北京 R1d ago
-
大模型算法研究员-MiMo CNY 500K-500KActive Learning | C++ | Curriculum learning | Data Preprocessing | Deep learningEntry-level Full Time北京1d ago
-
Miclaw-端云协同调度专家 (Hybrid AI Architect) CNY 240K-360KCloud Computing | Consistency protocols | Distributed Systems | Edge Computing | Language ModelsHybrid workSenior-level Full Time北京 R1d ago
-
Entry-level InternshipWuxi, CN1d ago
-
Big Data Engineer - Intern CNY 28K-50KAgile | Amazon EMR | Amazon S3 | Amazon Web Services | Apache AirflowEntry-level InternshipWuxi, CN1d ago
-
Big Data Engineer - Intern CNY 28K-50KAWS EMR | AWS S3 | Agile | Airflow | Apache HadoopFlexible clubs and activities | Mentorship opportunities | On-site meals | Team collaborationEntry-level InternshipWuxi, CN1d ago
-
AWS | Access Control | Apache Flink | Apache Hive | Apache KafkaFully paid annual leave | Housing fund | No business trip | Social insurance | Work-life balanceSenior-level Full TimeHangzhou, Zhejiang Sheng, China R1d ago
-
AI Infrastructure Software Engineer — CosmosLab CNY 360K-500KActivation checkpointing | C++ | CI/CD | CUDA | CheckpointingSenior-level Full TimeChina, Beijing1d ago
-
Lead Embedded Software Engineer CNY 360K-600KAgile | Analyzers | BLE | Bitbucket | Bluetooth Low EnergySenior-level Full TimeChina - Shenzhen Office1d ago
-
大模型算法工程师（Memory/RAG/意图识别） CNY 180K-360KARVR | Computer Vision | Data cleaning | Data labeling | Deep learningMid-level Full Time深圳2d ago
-
大模型算法实习生 CNY 25K-37KComputer Vision | Deep learning | Language Processing | Machine Learning | Multimodal LearningEntry-level Internship深圳2d ago