Senior Software Engineer I, Inference
Tasks
- Decompose work into milestones
- Define SLIs/SLOs
- Drive system architecture
- Execute rollback and traffic shift strategies
- Implement graceful degradation
- Implement inference performance optimizations
- Improve coding and testing standards
- Improve reliability through post incident actions
- Lead design reviews
- Mentor engineers
- Own multi service area
- Plan capacity for incidents
- Quantify optimization impact
- Review cross-team designs
- Tune autoscaling policies
Perks/Benefits
- 401k match
- Dental insurance
- Disability insurance
- Employee stock purchase program
- Flexible PTO
- Flexible childcare support
- Flexible spending account
- Health savings account
- Life insurance
- Long-term disability
- Medical insurance
- Mental wellness benefits
- Paid parental leave
- Short-term disability
- Tuition reimbursement
- Vision insurance
Skills/Tech-stack
Adaptive scheduling | Autoscaling | BF16 | C++ | CI/CD | CUDA | Caching | Cost Per | Cost Per Token Analytics | Cost per token | Distributed Systems | FP8 | GPU Interconnect | GPU Resource Isolation | Go | Graceful Degradation | Grafana | Inference Batching | KV cache | Kubernetes | Kv cache reuse | Microbatch Scheduling | Mixed Precision | NCCL | NUMA | Network Performance | OpenTelemetry | Prometheus | Python | RDMA | Ray Serve | Request Routing | Resource Isolation | Rollback | SHARP | SLI | SLO | Service reliability | Speculative decoding | Streaming token delivery | Tail Latency | TensorRT-LLM | Token analytics | Torchserve | Traffic shifting | Triton | VLLM
Education
Regions
Countries
States
Related jobs
-
Data Engineer II USD 84K-140KAgile | Amazon Redshift | Apache Airflow | CI/CD | Cloud ComputingMentorship | Professional developmentMid-level Full TimeBaltimore, Maryland, United States; McLean, Virginia, …2h ago
-
Databricks Engineering Consultant USD 97K-162KApache Spark | Data Governance | Data Lineage | Databricks | ELTMentorship programs | Professional development | Travel opportunitiesMid-level Full TimeRosslyn, Virginia, United States; Washington, District …2h ago
-
Sr Lead Software Engineer - Java, Python, AI/ML USD 170K-189KAWS | Algorithms | Auto-GPT | Automated testing | Cause analysisSenior-level Full TimeJersey City, NJ, United States3h ago
-
Senior LLM Engineer USD 195K-255KAI Foundry | AI Search | Azure AI | Azure AI Foundry | Azure AI SearchSenior-level Full TimeAustin, Texas | Fremont, California | …3h ago
-
Mid-level Full TimeUS-Georgia-Augusta4h ago
-
Automation | C# | C++ | CPU Utilization | Capacity PlanningDevelopment workshops | Hands-on learning opportunities | Industry exposure | Mentorship | Social eventsEntry-level InternshipSan Jose, California, United States4h ago
-
Machine Learning Engineer Graduate (E-Commerce Supply Chain & Logistics - LLM/Agent) - 2027 Start (PhD) USD 137K-246KAgents | Benchmarking | C++ | Deep learning | EvaluationEntry-level Full TimeSan Jose, California, United States4h ago
-
Machine Learning Engineer Graduate (E-Commerce Supply Chain & Logistics - LLM/Agent) - 2027 Start (PhD) USD 137K-246KAI Feedback | Agents | Benchmarking | C++ | DPOEntry-level Full TimeSeattle, Washington, United States4h ago
-
Agent evaluation | Classification | Clustering | Data Mining | Deep learningEntry-level Full TimeSeattle, Washington, United States4h ago
-
Apache Flink | Apache Hive | Apache Spark | Batch Processing | C++Entry-level Full TimeSan Jose, California, United States4h ago
-
Classification | Clustering | Data Mining | Deep learning | E-commerceEntry-level Full TimeSan Jose, California, United States4h ago
-
AI Workflow Orchestration | AI workflow | CD pipeline | CI Pipeline | CI/CDMid-level Full TimeSeattle, Washington, United States4h ago
-
Machine Learning Engineer (Technical Leadership) USD 213K-251KAB Testing | C plus plus | Data Mining | Data Pipelines | Deep learningSenior-level Full TimeMenlo Park, CA | New York, …5h ago
-
Embedded Software Engineer USD 147K-208KARM Cortex | ARM Cortex A | ARM Cortex-M | Asynchronous Interface | Boot loaderAI tools for development acceleration | Access to advanced prototyping hardware | Cross functional research environment | Flexible fast iteration | Ownership cultureSenior-level Full TimeRedmond, WA5h ago
-
Software Developer, Scaled Ops AI Acceleration Team USD 170K-209KAgent Orchestration | Artificial Intelligence | Backend Development | Bug tracking | ComplianceSenior-level Full TimeAustin, TX | Menlo Park, CA …5h ago
-
Research Engineer, Machine Perception (Project Aria) USD 141K-208K3D Computer Vision | C++ | Computer Vision | Data Pipelines | Deep learningOnsite work | Work authorization supportMid-level Full TimeRedmond, WA5h ago
-
Algorithms | Android | Android AppCompat | C# | C++Mid-level Full TimeWaterloo, ON, Canada; San Jose, CA, …5h ago
-
3D Perception | C++ | Camera Calibration | Computer Vision | Data StructuresSenior-level Full TimeSeattle, WA, USA; Mountain View, CA, …5h ago
-
Staff Software Engineer, Data Acquisition Infrastructure USD 207K-300KAPI Design | C++ | Data Structures | Data Structures and Algorithms | Distributed SystemsSenior-level Full TimeSan Jose, CA, USA5h ago
-
As-a-Service | C++ | Cloud Computing | Distributed Systems | Network SecuritySenior-level Full TimeSunnyvale, CA, USA5h ago
-
Staff Software Engineer, Applied AI, Model Quality USD 207K-300KData Processing | Data Storage | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeNew York, NY, USA5h ago
-
Software Engineer III, AI/ML GenAI, Search USD 147K-210KC++ | Data Processing | Debugging | Information Retrieval | LLMSenior-level Full TimeNew York, NY, USA5h ago
-
C++ | Cloud Computing | Data Processing | Debugging | Distributed ComputingSenior-level Full TimeMountain View, CA, USA5h ago
-
Senior Staff Software Engineer, AI/ML, Google Cloud USD 262K-364KAccelerated computing | Artificial Intelligence | Compute Engine | Data Processing | Data StorageSenior-level Full TimeSeattle, WA, USA; Sunnyvale, CA, USA5h ago
-
AI Inference | Benchmarking | C++ | Collective communication | Distributed SystemsSenior-level Full TimeMountain View, CA, USA5h ago