Backend Inference Runtime Engineer Graduate (AML Inference) - 2027 Start
San Jose, California, United States
USD 128K-256K Entry-level Full Time
Tasks
- Adapt inference engine to GPU and NPU hardware
- Benchmark with vLLM and TensorRT-LLM
- Design distributed parallel inference solutions
- Develop cache optimization strategies
- Handle load imbalance and parallel efficiency
- Implement inference framework performance optimizations
- Implement mixture of experts expert parallelism
- Implement operator fusion
- Implement pipeline parallelism
- Implement sequence parallelism
- Implement stream asynchronous scheduling
- Implement tensor parallelism
- Improve end to end GPU performance
- Increase single card inference throughput
- Optimize GPU memory access
- Optimize computing pipeline
- Optimize large model inference engine architecture
- Optimize multi card splitting and deployment
- Perform compilation optimization
- Reduce cross card communication overhead
- Reduce inference latency
Perks/Benefits
- 401k match
- Dental insurance
- Life insurance
- Long-term disability
- Medical insurance
- Paid Holidays
- Paid parental leave
- Paid personal time
- Paid sick days
- Short-term disability
- Vision insurance
- Wellbeing benefits
Skills/Tech-stack
Access Optimization | C# | C++ | CUDA | Computational Graph Optimization | Computational graph | Constant folding | Deep learning | Distributed Systems | GPU | GPU Performance | GPU memory | GPU performance analysis | Graph optimization | Memory Reuse | Memory access | Memory access optimization | Mixture of Experts | NPU | NVIDIA Nsight | Operator fusion | Performance Analysis | Pipeline parallelism | Profiling | Python | Quantization | Scheduling optimization | Sequence parallelism | Tensor Parallelism | Vectorization
Education
Regions
Countries
States
Cities
Related jobs
-
Senior Production Engineer - Applied Machine Learning USD 218K-480KAlerting | Auto Scaling | Auto rollback | C++ | CI/CD401k matching | Dental insurance | Disability coverage | Health insurance | Life insuranceSenior-level Full TimeSan Jose, California, United States4h ago
-
AI/LLM Network Software Development Engineer Graduate (High Speed Network) - 2026 Start (PhD) USD 162K-316KAI network | AI network optimization | C++ | Computer Networking | Congestion Control401k savings plan | Life insurance | Long-term disability | Medical, dental, and vision insurance | Paid HolidaysEntry-level Full TimeSan Jose, California, United States4h ago
-
Alerting | Automation | Bash | Capacity Planning | Change Management401k match | Life insurance | Long-term disability | Medical, dental, and vision insurance | Paid HolidaysSenior-level Full TimeSan Jose, California, United States4h ago
-
Algorithms | Apache Hive | Apache Spark | Big Data | C++401k savings plan | Company match 401k | Dental insurance | Life insurance | Long-Term Disability coverageMid-level Full TimeSeattle, Washington, United States4h ago
-
Algorithm Application Scientist - Large Model Applications - Global Frontier Tech Recruitment Program - 2027 Start (PHD) USD 218K-480KAgent Tool Calling | Deep learning | Elasticsearch | Fine Tuning | LangchainNone Full TimeSan Jose, California, United States4h ago
-
Tech Lead - Machine Learning Platform Engineer USD 254K-480KData Processing | Deep learning | Distributed Computing | Feature Engineering | Go401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeSan Jose, California, United States4h ago
-
LLM AIOps Development Engineer - Data Center Networking USD 162K-387KAgents | Anomaly Detection | Artificial Intelligence | BGP | CI/CDMid-level Full TimeSan Jose, California, United States4h ago
-
Vision Algorithm Engineer- Pico Lab- San Jose USD 218K-480KC++ | Computer Vision | Deep learning | Hardware acceleration | Image Processing401k match | Life insurance | Long-Term Disability coverage | Medical, dental, and vision insurance | Paid HolidaysMid-level Full TimeSan Jose, California, United States4h ago
-
Research Engineer / Scientist - Storage for LLM USD 162K-387KBatching | C++ | CUDA | CUDA Networking | CUDA aware Networking401k match | Conference attendance | Life insurance | Long-term disability | Medical/Dental/Vision insuranceMid-level Full TimeSan Jose, California, United States R4h ago
-
Video Algorithm Engineer - Multimedia Lab USD 218K-480KABR | AV1 | AVC | Adaptive bitrate | Adaptive bitrate streaming401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeSan Jose, California, United States4h ago
-
AI/LLM Network Software Development Engineer - San Jose USD 218K-480KAI network | AI network optimization | C++ | Computer Networking | Congestion Control401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeSan Jose, California, United States4h ago
-
C++ | CUDA | Checkpointing | Distributed Training | Fault ToleranceSenior-level Full TimeSan Jose, California, United States4h ago
-
Software Development Engineer-AI/LLM Network-Global Frontier Tech Research Program-2027 Start USD 218K-480KC++ | Computer Networking | Congestion Control | Go | MPI401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeSan Jose, California, United States4h ago
-
Algorithms | C++ | Compilers | Data Structures | DebuggingHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level Full TimeSeattle, Washington, United States4h ago
-
Data Engineer II USD 92K-176KAWS | Airflow | Azure | Azure DevOps | Azure Synapse401k matching | Dental insurance | Health insurance | Onsite fitness center | Relocation assistanceSenior-level Full TimeUS-Rhode Island-Providence4h ago
-
Senior Software Engineer- Metadata Storage USD 218K-387KC# | C++ | DPDK | Etcd | Go401k savings plan with company match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeSan Jose, California, United States4h ago
-
Video Codec Algorithm Modeling Engineer - Multimedia Lab USD 218K-480KAV1 | AVC H264 | C plus plus | C# | Coding Standards401k savings plan | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeSan Jose, California, United States4h ago
-
Software Development Engineer-AI/LLM Network-Global Frontier Tech Research Program-2027 Start (PhD) USD 207K-368KC# | C++ | Cause analysis | Computer Networking | Congestion Control401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceNone Full TimeSeattle, Washington, United States4h ago
-
Senior Research Engineer / Scientist - Storage for LLM USD 207K-368KAutoregressive decoding | Batched decoding | C++ | CUDA | CUDA aware Networking401k savings plan with company match | Conference attendance | Life insurance | Long-term disability | Medical/Dental/Vision insuranceSenior-level Full TimeSeattle, Washington, United States R4h ago
-
C# | C++ | Convolutional Neural Networks | Data Movement | Deep learning401k savings plan with company match | Life insurance | Long-Term Disability coverage | Medical/Dental/Vision insurance | Paid HolidaysMid-level Full TimeSan Jose, California, United States4h ago
-
LLM AIOps Development Engineer - Data Center Networking USD 207K-368KAgents | Anomaly Detection | BGP | CI/CD | Cause analysis401k matching | Life insurance | Long-term disability | Paid Holidays | Paid parental leaveMid-level Full TimeSeattle, Washington, United States4h ago
-
Benchmarking | C++ | Code Intelligence | Data Pipelines | Deep learningEntry-level Full TimeSan Jose, California, United States4h ago
-
Autoscaling | C++ | Computer networks | Concurrent programming | Container RuntimeHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level InternshipSan Jose, California, United States4h ago
-
C# | C++ | CPU architecture | Cache optimization | Code generation401k match | Dental insurance | Life insurance | Medical insurance | Paid HolidaysEntry-level Full TimeSan Jose, California, United States4h ago
-
Data Structures | Data Structures and Algorithms | Deep learning | Ensemble learning | Graph theoryHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level InternshipSeattle, Washington, United States4h ago