Senior Inference Runtime Engineer
Singapore, SG / Penang, MY / Taiwan
TWD 1900K-2500K (estimate) Senior-level Full Time
Tasks
- Apply pipeline parallelism
- Apply speculative decoding
- Apply tensor parallelism
- Collaborate with SRE for production improvements
- Create runtime playbooks
- Define safe default parameters
- Implement continuous batching
- Implement quantization strategies
- Improve long context serving
- Lead model onboarding for runtime selection
- Optimize prefill decode scheduling
- Optimize streaming smoothness
- Profile GPU memory bottlenecks
- Profile NCCL and network bottlenecks
- Profile proxy and frontend bottlenecks
- Profile tokenizer bottlenecks
- Set context length configurations
- Support model worker performance
- Tune KV cache behavior
- Tune TensorRT LLM style runtimes
- Tune vLLM style runtimes
Perks/Benefits
Skills/Tech-stack
CUDA | CUDA profiling | Continuous batching | Distributed inference | GPU Memory Optimization | GPU memory | Go | HBM Bandwidth | KV cache | LLM serving | Memory Optimization | Model Quantization | NCCL | Pipeline parallelism | Python | Speculative decoding | Streaming | Tensor Parallelism | TensorRT-LLM | Triton | VLLM
Education
N/A
Related jobs
-
Customer Engineer, Data Analytics, Google Cloud SGD 180K-221KCloud Architecture | Cloud platform | Data Governance | Data Lake | Data MigrationSenior-level Full TimeSingapore1d ago
-
VP / AVP, GenAI Engineer/Scientist, Innovation Group SGD 130K-200KAgent evaluation | Audit Logging | Bias Testing | Classification | Data PreparationExecutive-level Full TimeCentral Region (City Area), Singapore1d ago
-
Agentic AI | Audit Logging | Bias Testing | Calibration | Cause analysisExecutive-level Full TimeCentral Region (City Area), Singapore1d ago
-
VP1 Senior GenAI Engineer, GMET SGD 162K-205KAPI Design | Agentic Workflows | Containerization | Context Management | Distributed SystemsSenior-level Full TimeCentral Region (City Area), Singapore1d ago
-
Manager, Data Engineer, Group Technology & Ops SGD 115K-143KApache Flink | Apache Hive | Apache Iceberg | Apache Kafka | Apache NiFiMid-level Full TimeWest Region (City Area), Singapore1d ago
-
VP, Data Scientist, Enterprise AI, Innovation Group SGD 130K-261KAgentic AI | Agile project management | Boosting | Data Visualization | Deep learningExecutive-level Full TimeCentral Region (City Area), Singapore1d ago
-
CI/CD | Data Pipelines | DevOps | ETL | HadoopExecutive-level Full TimeCentral Region (City Area), Singapore1d ago
-
数据挖掘工程师 SGD 67K-78KAssociation Mining | Data Mining | Data Structures | Deep learning | ForecastingEntry-level Full Time新加坡1d ago
-
Mid-level Full TimeSingapore, Singapore2d ago
-
.NET | Agent Orchestration | Agent systems | App Service | AuthenticationMid-level Full TimeSingapore, Singapore2d ago
-
Control Theory | Domain Adaptation | Domain Randomization | Imitation Learning | IsaacGymFast iteration cycle | High visibility projects | Open source contributionsMid-level Full TimeSingapore2d ago
-
Machine Learning Engineer (Data Mining) - TikTok Shop SGD 143K-180KC++ | Data Mining | Decision Trees | Factorization machines | FlinkMid-level Full TimeSingapore2d ago
-
Agent systems | Cloud platform | Cost Per | Cost Per Request | CrewAIMid-level Full TimeSingapore2d ago
-
Data Quality | Data Validation | Data Visualization | Data pipeline | Deep learningMid-level Full TimeSingapore2d ago
-
Sr Advanced Software Engr (Data Scientist) SGD 160K-225KAzure Cloud | CI/CD | Computer Vision | Data Modeling | Deep learningSenior-level Full TimeSingapore2d ago
-
100G Ethernet | 200G Ethernet | 400G Ethernet | As-a-Service | BIOSSenior-level Contract Full TimeSingapore - Remote R2d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Career growth opportunities | Friendly work atmosphereMid-level Full TimeSingapore, Singapore2d ago
-
Bash | Cloud infrastructure | Cloud platform | Data Pipelines | Data ProcessingAsynchronous culture | Friendly work environment | Professional development opportunitiesMid-level Full TimeHsinchu, Taiwan2d ago
-
Research Engineer I (Computer Science / Artificial Intelligence / Data Science / Electrical Engineering) SGD 60K-108KBenchmarking | Deep learning | Distributed Training | Efficient Fine Tuning | Efficient InferenceMid-level Full TimeNTU Main Campus, Singapore2d ago
-
(Sr.) AI/ML engineer (TrenLife) TWD 1000K-1300KAWS | Agentic AI | Azure | Cloud platform | Computer VisionSenior-level Full TimeTaipei, Taiwan2d ago
-
AWS | Agile | Artificial Intelligence | Azure Databricks | Data GovernanceHands-on experience | Mentorship | TrainingEntry-level Full TimeSingapore, Singapore3d ago
-
AI machine learning | APIs | Cloud infrastructure | Content Management | DebuggingSenior-level Full TimeSingapore3d ago
-
Lead Engineer, Machine Learning SGD 149K-220KData pipeline | Deep learning | Fine Tuning | GPU Computing | JAXSenior-level Full TimeSingapore R3d ago
-
Machine Learning Engineer / Data Scientist (OCR/CV) SGD 166K-225KA/B | A/B Testing | B testing | CI/CD | Computer VisionMid-level Full TimeSingapore R3d ago
-
Software Engineer, ML Dev Enablement SGD 115K-140KAWS | C++ | DDP | Data loading | Distributed SystemsMid-level Full TimeSingapore, Central, Singapore3d ago