Senior Inference Engineer, AGI
Tasks
- Build offline inference systems for training RL and evaluation
- Build real time streaming inference runtime
- Build reward model and rollout generation pipelines
- Co design model architectures for serving
- Customize inference serving frameworks for streaming
- Establish latency throughput and cost benchmarking
- Implement and optimize inference path for multimodal models
- Implement continuous batching scheduling and preemption
- Implement multi GPU tensor parallel inference
- Meet sub second latency under concurrent load
- Own real time multimodal inference stack
- Support train serve consistency for RL and evaluation
Perks/Benefits
- 401k matching
- Dental insurance
- Health insurance
- Mental health support
- Paid time off
- Parental leave
- Vision insurance
Skills/Tech-stack
Attention | Autoregressive decoding | CUDA | CUDA kernels | Continuous batching | Deep learning | Distributed Training | GPU Performance | GPU Performance Optimization | Inference Optimization | KV cache | Multimodal Learning | NCCL | Operator fusion | Performance optimization | PyTorch | Quantization | RLAIF | RLHF | Reinforcement Learning | Speculative decoding | Tensor Parallelism | TensorRT-LLM | Transformers | Triton | VLLM
Education
Regions
Countries
States
Cities
Related jobs
-
Machine Learning System Scheduling Engineer Graduate (Applied Machine Learning) - 2027 Start USD 128K-256KAlgorithms | CUDA | Cloud Storage | Computer Science | ContainerizationEntry-level Full TimeSan Jose, California, United States4h ago
-
AI/LLM Network Software Development Engineer Graduate (High Speed Network) - 2027 Start (PhD) USD 162K-316KAI network | AI network optimization | C# | C++ | Computer Networking401k with company match | Dental insurance | Life insurance | Long-term disability | Medical insuranceEntry-level Full TimeSan Jose, California, United States4h ago
-
ARM Cortex | ARM Cortex M4 | AV1 | AVC | C++401-k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceEntry-level Full TimeSan Diego, California, United States4h ago
-
AUC Optimization | Apache Flink | Apache Hadoop | Apache Spark | Big Data401k savings plan | Dental insurance | Life insurance | Long-term disability | Medical insuranceEntry-level Full TimeSan Jose, California, United States4h ago
-
Algorithms | CUDA | Computer Architecture | Data Structures | Deep Learning Training401k match | Life insurance | Long-term disability | Medical/Dental/Vision insurance | Paid HolidaysEntry-level Full TimeSan Jose, California, United States5h ago
-
Machine Learning Engineer USD 170K-250KAzure ML | Azure ML Endpoints | Azure Machine Learning | Bayesian Methods | Boosted TreesSenior-level Full TimeAustin, Texas | Fremont, California | …5h ago
-
Data Structures | Data Structures and Algorithms | Deep learning | Ensemble learning | Graph theory401k match | Dental insurance | Health insurance | Life insurance | Long-term disabilityEntry-level Full TimeSeattle, Washington, United States5h ago
-
Deep Neural Networks | Ensemble learning | Graph theory | Hadoop | HiveHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level InternshipSeattle, Washington, United States5h ago
-
Data Structures | Data Structures and Algorithms | Deep learning | Ensemble Methods | Graph theory401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceEntry-level Full TimeSeattle, Washington, United States5h ago
-
Data Querying | Data Structures | Deep Neural Networks | Ensemble learning | Graph theory401k matching | Dental insurance | Life insurance | Long-term disability | Medical insuranceEntry-level Full TimeSan Jose, California, United States5h ago
-
AWS | Azure | Azure ML | C++ | CUDAEntry-level Full TimeSeattle, Washington, United States5h ago
-
Machine Learning Engineer, AI Coding Tools USD 162K-387KC++ | CUDA | CUDA Deployment Optimization | Deep learning | Deployment optimization401k savings plan | Dental insurance | Life insurance | Medical insurance | Paid HolidaysMid-level Full TimeSan Jose, California, United States5h ago
-
Computer Vision | Data Processing | Deep learning | Language Models | Large Language ModelsSenior-level Full TimeSan Jose, California, United States5h ago
-
C++ | Computer Vision | Deep learning | End to End | End to End Machine Learning Pipelines401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeSan Jose, California, United States5h ago
-
Production Engineer - Applied Machine Learning USD 162K-387KAlerting | Auto Scaling | Auto rollback | C++ | CI/CD401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceMid-level Full TimeSan Jose, California, United States5h ago
-
Alignment for Large Language Models | C++ | Deep learning | Fine Tuning | GoHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level InternshipSan Jose, California, United States5h ago
-
Machine Learning Engineer - AI Compiler Optimization USD 184K-287KCo-design | Code generation | GPU compilation | Hardware-Software Co-design | Hardware/softwareSenior-level Full TimeSan Jose, California, United States5h ago
-
Machine Learning Engineer - Orchestration USD 162K-250KApache Flink | Apache Mesos | Auto Scaling | Celery | Deep learningMid-level Full TimeSan Jose, California, United States5h ago
-
Data Processing | Deep learning | Distributed Computing | Feature Engineering | Golang401k plan with company match | Dental insurance | Health insurance | Life insurance | Long-Term Disability coverageMid-level Full TimeSan Jose, California, United States5h ago
-
Anomaly Detection | Apache Flink | Apache Spark | C++ | ClassificationHealth insurance | Housing allowance | Life insurance | Paid Holidays | Paid sick timeEntry-level InternshipSan Jose, California, United States5h ago
-
Machine Learning Engineer - Inference USD 162K-387KDistributed Systems | GPU Acceleration | High Performance | High-Performance Computing | JAX401k savings plan match | Life insurance | Medical, dental, and vision insurance | Paid Holidays | Paid parental leaveMid-level Full TimeSan Jose, California, United States5h ago
-
AI/LLM Network Software Development Engineer USD 207K-368KC# | C++ | Computer Networking | Congestion Control | Go401k match | Dental insurance | Health insurance | Life insurance | Long-term disabilityMid-level Full TimeSeattle, Washington, United States5h ago
-
Software Engineer - Applied Machine Learning, Engine USD 128K-316KAWS | Alerting | Algorithms | Azure | CMDB401k match | Dental insurance | Health insurance | Life insurance | Long-term disabilityEntry-level Full TimeSan Jose, California, United States5h ago
-
Senior Production Engineer - Applied Machine Learning USD 218K-480KAlerting | Auto Scaling | Auto rollback | C++ | CI/CD401k matching | Dental insurance | Disability coverage | Health insurance | Life insuranceSenior-level Full TimeSan Jose, California, United States5h ago
-
AI/LLM Network Software Development Engineer Graduate (High Speed Network) - 2026 Start (PhD) USD 162K-316KAI network | AI network optimization | C++ | Computer Networking | Congestion Control401k savings plan | Life insurance | Long-term disability | Medical, dental, and vision insurance | Paid HolidaysEntry-level Full TimeSan Jose, California, United States5h ago