Senior Software Engineer II, Inference
Tasks
- Apply graceful degradation
- Decompose work into milestones
- Define SLIs and SLOs
- Drive system architecture
- Execute rollback and traffic shift strategies
- Implement model serving optimizations
- Improve coding and testing standards
- Lead design reviews
- Lead multi service reliability improvements
- Mentor engineers
- Own post incident actions
- Plan capacity and autoscaling
- Quantify performance impact
- Review cross-team designs
Perks/Benefits
- 401k match
- Employee stock purchase program
- Flexible PTO
- Flexible spending account
- Health savings account
- Life insurance
- Medical/Dental/Vision insurance
- Paid parental leave
- Tuition reimbursement
- Tuition support
Skills/Tech-stack
Autoscaling | BF16 | C++ | CI/CD | CUDA | Caching | Capacity Planning | Distributed Systems | FP8 | GPU Computing | Go | Grafana | Incident Management | Inference Batching | KV cache | Kubernetes | Latency optimization | Microbatching | Mixed Precision | NCCL | NUMA | Networking | OpenTelemetry | Performance Engineering | Prometheus | Python | RDMA | Ray Serve | Reliability Engineering | SLI | SLO | Speculative decoding | Streaming token delivery | TensorRT-LLM | Throughput Optimization | Torchserve | Triton | VLLM
Education
N/A
Regions
Countries
States
Related jobs
-
Senior Product Engineer, Machine Learning Accelerators USD 144K-209KBash | Bench testing | Board assembly | Bring-up | Cause analysisSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Software Engineer, Storage Everywhere, Google Cloud USD 147K-211KBack-end | Back-end development | C++ | Cloud Platforms | Data StructuresMid-level Full TimeSunnyvale, CA, USA3h ago
-
Senior Staff Software Engineer, Torch TPU USD 262K-365KAlgorithms | C++ | Cloud Computing | Computer Vision | Data ProcessingSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Staff Software Engineer, TPU Performance USD 207K-300KCUDA | Code generation | Compiler optimization | Data Processing | Data StructuresSenior-level Full TimeMountain View, CA, USA3h ago
-
Senior Software Engineer, AI/ML GenAI, Google Cloud USD 174K-252KC plus plus | Computer Vision | Data Processing | Debugging | Distributed ComputingSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Senior Software Engineer, AI/ML GenAI, Google Play USD 174K-252KAndroid | C++ | Computer Vision | Data Processing | Data StorageSenior-level Full TimeMountain View, CA, USA3h ago
-
Staff Software Engineer, AI/ML GenAI, Google Cloud USD 207K-300KAlgorithms | C++ | Cloud infrastructure | Computer Vision | Data ProcessingSenior-level Full TimeSunnyvale, CA, USA; San Francisco, CA, …3h ago
-
Software Engineer Manager, Embedded Systems and Firmware USD 207K-300KARM | Bootloader | C# | C++ | Computer ArchitectureSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Senior Software Engineer, AI/ML, Google Workspace USD 174K-252KC++ | Data Processing | Debugging | Language Processing | ML InfrastructureSenior-level Full TimeNew York, NY, USA; Sunnyvale, CA, …3h ago
-
Senior Software Engineer, Data Cloud Memorystore USD 174K-252KC++ | Caching | Cloud Computing | Cloud Networking | Cloud platformSenior-level Full TimeKirkland, WA, USA3h ago
-
RTL Design Engineer, Machine Learning Accelerators USD 138K-198KASIC design | Code review | High Performance | High-Performance Design | Low powerMid-level Full TimeSunnyvale, CA, USA3h ago
-
C# | C++ | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA3h ago
-
Software Engineer III, AI/ML, Google Fi USD 147K-211KData Processing | Debugging | Distributed Systems | Java | Language ProcessingSenior-level Full TimeMountain View, CA, USA; San Francisco, …3h ago
-
Senior Software Developer, Embedded Systems/Firmware, XR USD 174K-252KAlgorithms | Android | Android HAL | Bootloader | C#Senior-level Full TimeSan Jose, CA, USA; Waterloo, ON, …3h ago
-
Cloud & Data Engineer USD 83K-187KAWS | Azure | Big Data | CI/CD | Container Security401k savings plan | Disability insurance | Employee assistance program | Flexible vacation | Life insuranceEntry-level Full TimeLexington, MA, US6h ago
-
AWS Data Engineer USD 118K-178KAWS CloudFormation | AWS Data | AWS Data Migration | AWS Data Migration Service | AWS EMR401k matching | Dental insurance | Education assistance program | Health insurance | Holiday payMid-level Full TimeWheeling, IL, US7h ago
-
AI Solution Principal Systems Development Engineer USD 148K-222KCluster management | Containerization | GPU Computing | Generative AI | Hardware/Software Integration401k plan | Choice of medical coverage | Fitness reimbursement | Generous Time Off Programs | Team member discounts on Dell productsSenior-level Full TimeAustin, Texas, United States, United States14h ago
-
Lead Cloud Data Engineer USD 116K-155KAWS Glue | AWS IAM | AWS Lake Formation | Amazon Athena | Amazon AuroraDental insurance | Hybrid work | Life insurance | Medical insurance | Paid time offSenior-level Full TimeFL - Saint Petersburg - 880 …14h ago
-
AI Search | Azure AI | Azure AI Search | CI/CD | Chroma401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States14h ago
-
Junior Applied AI Engineer- HRIS USD 105K-140KAWS | Azure | Cloud Platforms | Cloud platform | Data Engineering401k | Dental insurance | Disability insurance | Leave Program | Life insuranceEntry-level Full TimeSunnyvale, CA, United States15h ago
-
API Security | AWS | Adversarial inputs | Agent Frameworks | Application SecurityCiti Bike membership | Flexible paid time off | Gym membership | Health and mental health services | Home office stipendSenior-level Full TimeSan Francisco, California, United States; Seattle, …15h ago
-
Applied AI ML, Senior Associate USD 171K-245KAgentic AI | Cloud services | Continuous integration | Data Analysis | Data PipelinesBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeHouston, TX, United States16h ago
-
APIs | Artificial Intelligence | Automation | Decision support | Generative AI401k | Cafe Amenities | Fitness Center | Health insurance | Recreational facilitiesEntry-level Full TimeBroomfield, CO, United States17h ago
-
Agentic AI | Data Analysis | Data Pipelines | Financial Data | Financial Data AnalysisBackup childcare | Financial coaching | Health care coverage | Mental health support | On Site Health Wellness CentersEntry-level Full TimeJersey City, NJ, United States17h ago
-
Applied AI Engineer (AI Adoption) USD 200K-250KAI Evaluation | Agent Design | Automation | LLM APIs | Machine LearningDental insurance | Group pension scheme | Income protection | Life assurance | Paid HolidaysMid-level Full TimeNew York, Chicago, London17h ago