Senior Inference Runtime Engineer
Singapore, SG / Penang, MY / Taiwan
TWD 1900K-2500K (estimate) Senior-level Full Time
Tasks
- Apply pipeline parallelism
- Apply speculative decoding
- Apply tensor parallelism
- Collaborate with SRE for production improvements
- Create runtime playbooks
- Define safe default parameters
- Implement continuous batching
- Implement quantization strategies
- Improve long context serving
- Lead model onboarding for runtime selection
- Optimize prefill decode scheduling
- Optimize streaming smoothness
- Profile GPU memory bottlenecks
- Profile NCCL and network bottlenecks
- Profile proxy and frontend bottlenecks
- Profile tokenizer bottlenecks
- Set context length configurations
- Support model worker performance
- Tune KV cache behavior
- Tune TensorRT LLM style runtimes
- Tune vLLM style runtimes
Perks/Benefits
Skills/Tech-stack
CUDA | CUDA profiling | Continuous batching | Distributed inference | GPU Memory Optimization | GPU memory | Go | HBM Bandwidth | KV cache | LLM serving | Memory Optimization | Model Quantization | NCCL | Pipeline parallelism | Python | Speculative decoding | Streaming | Tensor Parallelism | TensorRT-LLM | Triton | VLLM
Education
N/A
Related jobs
-
AI Customer Engineer, Telecommunications, Google Cloud SGD 160K-220KAI Agent Development Kit | Agent Development Kit | As-a-Service | Audit Logging | Cloud NetworkingConference participation | Travel opportunitiesSenior-level Full TimeSingapore3h ago
-
Data Engineer, ANZSA SGD 150K-210KAPI Integration | Amazon Redshift | CI/CD | Data Governance | Data ModelingMid-level Full TimeSingapore6h ago
-
Lead Software Engineer, Perception (Robotics) SGD 155K-238K3D Perception | C++ | Camera | Camera Calibration | Cause analysisBirthday leave | FlexWork | Insurance | Medical insurance | Parental leaveSenior-level Full TimeSingapore, Singapore9h ago
-
AWS | AWS CodeDeploy | AWS ECS | Agile | Apache HudiSenior-level Full TimeTaipei, Taiwan14h ago
-
Mid-level Full TimeSingapore1d ago
-
Software Engineer, Machine Learning SGD 74K-74KBias Mitigation | Data Structures | Ethical AI | Graph theory | IndexingSenior-level Full TimeSingapore1d ago
-
Senior LLM Inference Performance & Evaluation Engineer TWD 1700K-2300KBenchmarking | CI/CD | Concurrency | Dashboards | GPU UtilizationDevelopment opportunities | Inclusive environment | Mentoring | Training opportunitiesSenior-level Full TimeSingapore, SG / Penang, MY / …1d ago
-
Cloud Storage Engineer TWD 1700K-2500KBeeGFS | CI/CD | Caching | Capacity Planning | CephAutonomy | Fast growth learning opportunities | Inclusive environment | Training and mentoring | Welfare benefitsSenior-level Full TimeSingapore, SG / Penang, MY / …1d ago
-
Machine Learning & Data Engineering Intern SGD 36K-48KC++ | Data Modeling | Data analytics | Machine Learning | PythonCommunity volunteer opportunities | Gym access | On site recreation facilities | Yoga classes | Zumba classesEntry-level InternshipShugart, SG1d ago
-
AI Integration Engineer SGD 60K-108KAPI Key | Audit Logging | BACnet | Code Integration | EncryptionFlexible working arrangements | Professional developmentMid-level Full TimeSJ Campus, Singapore1d ago
-
Entry-level Full TimeTaichung - Fab 16, Taiwan1d ago
-
Domain Randomization | Imitation Learning | IsaacGym | IsaacLab | JAXFast feedback cycles | High impact work on real robots | Open source contributionsMid-level Full TimeSingapore2d ago
-
C++ | Localization | Mapping | Motion Planning | NavigationCompetitive compensation | EquitySenior-level Full TimeSingapore2d ago
-
Autonomy | C++ | Contact Rich Control | Control Theory | Data collectionCompetitive compensation | Equity | Open source contributionsMid-level Full TimeSingapore2d ago
-
AI Engineer (GenAI / RAG / LangGraph / Python) SGD 74K-74KAgent systems | Anthropic Claude | Azure OpenAI | CI/CD | ChromaSenior-level Contract Full TimeSingapore, Singapore, Singapore2d ago
-
Classifier Modeling | Data Mining | Data Structures | Data Structures and Algorithms | Machine LearningSenior-level Full TimeSingapore3d ago
-
Tech Expert/Backend Engineer - Global Live (LLM) SGD 150K-190KAmazon RDS | Big Data | C++ | Golang | JavaSenior-level Full TimeSingapore3d ago
-
C++ | Cloud Computing | Data Modeling | Data Pipelines | ETLSenior-level Full TimeSingapore3d ago
-
AWS | Algorithms | Apache Airflow | Apache Pinot | Apache SparkSenior-level Full TimeTaipei, Taiwan; Tokyo, Japan3d ago
-
Senior Databricks Consulting Engineer SGD 175K-221KAWS | Apache Kafka | Apache Spark | Azure | Cloud ArchitectureContinuous learning and certification support | Remote friendly working environmentSenior-level Full TimeSingapore3d ago
-
APIs | Azure | Azure Data | Azure Data Explorer | Azure DevOpsOnsite workMid-level Full TimeSingapore, Singapore4d ago
-
.NET | AKS | API Integration | Agent Orchestration | Agent systemsSenior-level Full TimeSingapore, Singapore4d ago
-
AWS | Anomaly Detection | Anomaly detection models | Azure | ClassificationSenior-level Full TimeSingapore, Singapore4d ago
-
Senior-level Full TimeSingapore, Singapore4d ago
-
Senior AI Engineer USD 200K-300KCost Optimization | Fine Tuning | GPU Monitoring | GPU scheduling | Hugging FaceDental insurance | Dependent care options | Flexible spending account | Flexible vacation | Group term life insuranceSenior-level Full TimeChicago; Hong Kong; London; New York; …4d ago