Backend Inference Runtime Engineer Graduate (AML Inference) - 2027 Start
San Jose, California, United States
USD 128K-256K Entry-level Full Time
Tasks
- Adapt inference engine for GPU NPU hardware
- Benchmark against TensorRT-LLM
- Benchmark against vLLM
- Design distributed parallel inference solutions
- Implement Mixture of Experts parallelism
- Implement compilation optimization
- Implement operator fusion
- Implement pipeline parallelism
- Implement sequence parallelism
- Implement tensor parallelism
- Iterate large model inference engine architecture
- Optimize GPU inference throughput
- Optimize GPU memory access
- Optimize cache performance
- Optimize computing pipeline
- Reduce inference latency
- Schedule GPU streams asynchronously
Perks/Benefits
- N/A
Skills/Tech-stack
Asynchronous execution | Benchmarking | CUDA | Cache optimization | Compilation Optimization | Distributed Systems | GPU Computing | GPU Memory Optimization | GPU memory | Machine Learning | Memory Optimization | Mixture of Experts | NPU | Operator fusion | Parallel Computing | Pipeline parallelism | Sequence parallelism | Stream programming | Tensor Parallelism
Regions
Countries
States
Cities
Related jobs
-
Algorithms | Data Structures | Debugging | Java | JavaScriptOn-site work | Top Secret clearanceSenior-level Full TimeTysons, VA4h ago
-
Solution Architect/Data Engineer USD 150K-170KAPI Design | API Integration | Apache Kafka | Apache Spark | Artificial Intelligence401k match | Dental insurance | Medical insurance | Paid time off | Professional development and career advancementSenior-level Full TimeUnited States-Illinois-Wood Dale7h ago
-
AI/LLM Network Software Development Engineer Graduate (High Speed Network) - 2027 Start (PhD) USD 162K-316KAI communication | Distributed Systems | High Availability | Monitoring | Network DiagnosticsEntry-level Full TimeSan Jose, California, United States8h ago
-
Alerting | Canary Deployment | Distributed Systems | High Availability | Machine LearningEntry-level Full TimeSan Jose, California, United States8h ago
-
C++ | CUDA | CUDA toolkit | Code generation | Computation SchedulingEntry-level Full TimeSan Jose, California, United States8h ago
-
Software Engineer Graduate (Data-Intelligent Creation-AI Platform-Global Vision Engineering) - 2027 Start USD 128K-316KAutomation | CI/CD | Computer Vision | Generative AI | Language ProcessingEntry-level Full TimeSan Jose, California, United States8h ago
-
Applied Machine Learning Production Engineer Graduate (AML-Production Engineer) - 2027 Start USD 128K-256KAutomation | C++ | Distributed Systems | Linux | MonitoringMentorship | Professional developmentEntry-level Full TimeSan Jose, California, United States8h ago
-
Anomaly Detection | Data Pipelines | LLM | Language Processing | Log AnalysisEntry-level InternshipSan Jose, California, United States8h ago
-
Big Data | C++ | Data Pipelines | Distributed Storage | JavaEntry-level Full TimeSeattle, Washington, United States8h ago
-
Advertising | Big Data | C++ | Data Pipelines | Distributed StorageEntry-level Full TimeSeattle, Washington, United States8h ago
-
Machine Learning Engineer Graduate (E-Commerce Recommendation/Search Alliance) - 2027 Start USD 115K-177KAlgorithms | Big Data | Big Data Engineering | Data Engineering | Data StructuresEntry-level Full TimeSan Jose, California, United States8h ago
-
Machine Learning Engineer Graduate (E-Commerce Recommendation/Search Alliance) - 2027 Start (PhD) USD 129K-237KBig Data | Big Data Engineering | Data Engineering | Data Structures | Data structures algorithmsEntry-level Full TimeSan Jose, California, United States8h ago
-
Machine Learning Engineer Graduate (E-Commerce Recommendation/Search Alliance) - 2027 Start USD 115K-177KAlgorithms | Big Data | Big Data Engineering | Data Engineering | Data StructuresEntry-level Full TimeSeattle, Washington, United States8h ago
-
Machine Learning Engineer Graduate (E-Commerce Recommendation/Search Alliance) - 2027 Start (PhD) USD 129K-237KAlgorithms | Big Data | Big Data Engineering | Data Engineering | Data StructuresEntry-level Full TimeSeattle, Washington, United States8h ago
-
Machine Learning Engineer Intern (E-Commerce Recommendation/Search Alliance) - 2027 Start (PhD) USD 129K-237KAlgorithms | Big Data | Data Structures | Distributed Computing | Language ModelsCommunity building events | Internship | Professional development eventsEntry-level InternshipSeattle, Washington, United States8h ago
-
Algorithms | C plus plus | Causal Inference | Data Structures | DebiasingEntry-level Full TimeSan Jose, California, United States8h ago
-
Machine Learning Engineer Graduate (E-Commerce Recommendation Live) - 2027 Start (PhD) USD 129K-246KAdvertising | Algorithms | C++ | Causal Inference | Contrastive LearningEntry-level Full TimeSeattle, Washington, United States8h ago
-
C++ | Causal Inference | Contrastive Learning | Debiasing | Deep learningEntry-level Full TimeSeattle, Washington, United States8h ago
-
Machine Learning Digital Design Engineer USD 142K-203KASIC design | CDC | Clock Domain | Clock Domain Crossing | Computer ArchitectureSenior-level Full TimeSunnyvale, CA9h ago
-
Ads ML Developer Experience Tech Lead USD 174K-220KAI Agents | Agent Orchestration | Artificial Intelligence | Bias Mitigation | Build systemsSenior-level Full TimeSunnyvale, CA R9h ago
-
C++ | Compute Allocation | Dashboarding | Data Visualization | Machine LearningMid-level Full TimeMountain View, CA, USA; New York, …9h ago
-
Senior Software Engineer, Generative AI, gUP USD 174K-252KA/B | A/B Testing | B testing | Code review | Computer Vision24x7 support rotationSenior-level Full TimeSunnyvale, CA, USA9h ago
-
Software Engineer III, BigQuery USD 147K-210KArtificial Intelligence | BigQuery | Cloud Run | Data Storage | Data StructuresSenior-level Full TimeKirkland, WA, USA9h ago
-
Software Engineer, Supply Chain USD 147K-210KC++ | CSS | Cloud platform | Data Analysis | Data StructuresMid-level Full TimeSunnyvale, CA, USA9h ago
-
Software Engineer, Enterprise AI Capabilities USD 174K-252KAgentic AI | Cloud Computing | Distributed Systems | GCP | Generative AIMid-level Full TimeSan Jose, CA, USA9h ago