Senior Inference Engineer, AGI
Tasks
- Build offline inference systems for training RL and evaluation
- Build real time streaming inference runtime
- Build reward model and rollout generation pipelines
- Co design model architectures for serving
- Customize inference serving frameworks for streaming
- Establish latency throughput and cost benchmarking
- Implement and optimize inference path for multimodal models
- Implement continuous batching scheduling and preemption
- Implement multi GPU tensor parallel inference
- Meet sub second latency under concurrent load
- Own real time multimodal inference stack
- Support train serve consistency for RL and evaluation
Perks/Benefits
- 401k matching
- Dental insurance
- Health insurance
- Mental health support
- Paid time off
- Parental leave
- Vision insurance
Skills/Tech-stack
Attention | Autoregressive decoding | CUDA | CUDA kernels | Continuous batching | Deep learning | Distributed Training | GPU Performance | GPU Performance Optimization | Inference Optimization | KV cache | Multimodal Learning | NCCL | Operator fusion | Performance optimization | PyTorch | Quantization | RLAIF | RLHF | Reinforcement Learning | Speculative decoding | Tensor Parallelism | TensorRT-LLM | Transformers | Triton | VLLM
Education
Regions
Countries
States
Cities
Related jobs
-
ARM Cortex | ARM Cortex M4 | AV1 | AVC | C++401-k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceEntry-level Full TimeSan Diego, California, United States4h ago
-
Senior Production Engineer - Applied Machine Learning USD 218K-480KAlerting | Auto Scaling | Auto rollback | C++ | CI/CD401k matching | Dental insurance | Disability coverage | Health insurance | Life insuranceSenior-level Full TimeSan Jose, California, United States5h ago
-
AI/LLM Network Software Development Engineer Graduate (High Speed Network) - 2026 Start (PhD) USD 162K-316KAI network | AI network optimization | C++ | Computer Networking | Congestion Control401k savings plan | Life insurance | Long-term disability | Medical, dental, and vision insurance | Paid HolidaysEntry-level Full TimeSan Jose, California, United States5h ago
-
Algorithms | Apache Hive | Apache Spark | Big Data | C++401k savings plan | Company match 401k | Dental insurance | Life insurance | Long-Term Disability coverageMid-level Full TimeSeattle, Washington, United States5h ago
-
Algorithm Application Scientist - Large Model Applications - Global Frontier Tech Recruitment Program - 2027 Start (PHD) USD 218K-480KAgent Tool Calling | Deep learning | Elasticsearch | Fine Tuning | LangchainNone Full TimeSan Jose, California, United States5h ago
-
Tech Lead - Machine Learning Platform Engineer USD 254K-480KData Processing | Deep learning | Distributed Computing | Feature Engineering | Go401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeSan Jose, California, United States5h ago
-
LLM AIOps Development Engineer - Data Center Networking USD 162K-387KAgents | Anomaly Detection | Artificial Intelligence | BGP | CI/CDMid-level Full TimeSan Jose, California, United States5h ago
-
Vision Algorithm Engineer- Pico Lab- San Jose USD 218K-480KC++ | Computer Vision | Deep learning | Hardware acceleration | Image Processing401k match | Life insurance | Long-Term Disability coverage | Medical, dental, and vision insurance | Paid HolidaysMid-level Full TimeSan Jose, California, United States5h ago
-
Research Engineer / Scientist - Storage for LLM USD 162K-387KBatching | C++ | CUDA | CUDA Networking | CUDA aware Networking401k match | Conference attendance | Life insurance | Long-term disability | Medical/Dental/Vision insuranceMid-level Full TimeSan Jose, California, United States R5h ago
-
Video Algorithm Engineer - Multimedia Lab USD 218K-480KABR | AV1 | AVC | Adaptive bitrate | Adaptive bitrate streaming401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeSan Jose, California, United States5h ago
-
AI/LLM Network Software Development Engineer - San Jose USD 218K-480KAI network | AI network optimization | C++ | Computer Networking | Congestion Control401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeSan Jose, California, United States5h ago
-
C++ | CUDA | Checkpointing | Distributed Training | Fault ToleranceSenior-level Full TimeSan Jose, California, United States5h ago
-
Software Development Engineer-AI/LLM Network-Global Frontier Tech Research Program-2027 Start USD 218K-480KC++ | Computer Networking | Congestion Control | Go | MPI401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeSan Jose, California, United States5h ago
-
Algorithms | C++ | Compilers | Data Structures | DebuggingHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level Full TimeSeattle, Washington, United States5h ago
-
Video Codec Algorithm Modeling Engineer - Multimedia Lab USD 218K-480KAV1 | AVC H264 | C plus plus | C# | Coding Standards401k savings plan | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeSan Jose, California, United States5h ago
-
Software Development Engineer-AI/LLM Network-Global Frontier Tech Research Program-2027 Start (PhD) USD 207K-368KC# | C++ | Cause analysis | Computer Networking | Congestion Control401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceNone Full TimeSeattle, Washington, United States5h ago
-
Senior Research Engineer / Scientist - Storage for LLM USD 207K-368KAutoregressive decoding | Batched decoding | C++ | CUDA | CUDA aware Networking401k savings plan with company match | Conference attendance | Life insurance | Long-term disability | Medical/Dental/Vision insuranceSenior-level Full TimeSeattle, Washington, United States R5h ago
-
C# | C++ | Convolutional Neural Networks | Data Movement | Deep learning401k savings plan with company match | Life insurance | Long-Term Disability coverage | Medical/Dental/Vision insurance | Paid HolidaysMid-level Full TimeSan Jose, California, United States5h ago
-
LLM AIOps Development Engineer - Data Center Networking USD 207K-368KAgents | Anomaly Detection | BGP | CI/CD | Cause analysis401k matching | Life insurance | Long-term disability | Paid Holidays | Paid parental leaveMid-level Full TimeSeattle, Washington, United States5h ago
-
Benchmarking | C++ | Code Intelligence | Data Pipelines | Deep learningEntry-level Full TimeSan Jose, California, United States5h ago
-
Autoscaling | C++ | Computer networks | Concurrent programming | Container RuntimeHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level InternshipSan Jose, California, United States5h ago
-
C# | C++ | CPU architecture | Cache optimization | Code generation401k match | Dental insurance | Life insurance | Medical insurance | Paid HolidaysEntry-level Full TimeSan Jose, California, United States5h ago
-
Data Structures | Data Structures and Algorithms | Deep learning | Ensemble learning | Graph theoryHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level InternshipSeattle, Washington, United States5h ago
-
Research Engineer / Scientist -AI for Databases USD 162K-387KAWS | Architecture Search | Artificial Intelligence | Automated Database Tuning | Azure401k match | Conference attendance | Dental insurance | Life insurance | Long-term disabilityMid-level Full TimeSan Jose, California, United States R5h ago
-
AI/LLM Network Software Development Engineer Graduate (High Speed Network) - 2026 Start (PhD) USD 153K-300KAI network | AI network optimization | C++ | Computer networks | Congestion Control401k match | Life insurance | Long-Term Disability coverage | Medical/Dental/Vision insurance | Paid HolidaysEntry-level Full TimeSeattle, Washington, United States5h ago