Senior Software Engineer, Inference
Tasks
- Decompose multi-service work into milestones
- Define and own SLIs and SLOs
- Design graceful degradation strategies
- Drive post incident actions
- Implement inference optimizations
- Implement rollback and traffic shift plans
- Improve coding and testing standards
- Lead architecture and design reviews
- Mentor engineers and review designs
- Own GPU resource isolation
- Own cost per token analytics
- Own request routing and adaptive scheduling
- Perform capacity planning and autoscaling
- Quantify performance impact
Perks/Benefits
- 401k employer match
- Company paid life insurance
- Employee stock purchase program
- Flexible PTO
- Flexible spending account
- Health savings account
- Long-term disability
- Paid Insurance
- Paid parental leave
- Short-term disability
- Tuition reimbursement
Skills/Tech-stack
BF16 | C++ | CI/CD | CUDA | CUDA kernels | FP8 | Go | Grafana | KV cache | Kubernetes | Machine Learning | Mixed Precision | NCCL | NUMA | OpenTelemetry | Prometheus | Python | RDMA | Ray Serve | Speculative decoding | Streaming | TensorRT-LLM | Torchserve | Triton | VLLM
Education
N/A
Regions
Countries
States
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R3d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R16d ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R6h ago
-
Senior-level Full TimeMountain View, CA8h ago
-
Staff Software Engineer, AI Integrations USD 170K-240KAgile | Alerting | Amazon Web Services | C++ | CSSSenior-level Full TimeMountain View, CA8h ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-204KAPI Testing | Cypher | Data Processing | DataOps | DevOpsCompetitive benefits | Growth opportunity | Remote work | TravelSenior-level Full TimeReston, VA, United States R18h ago
-
AWS | AWS CloudFormation | Amazon ECS | Automation | CI/CD401-k plan | Company match | Employee stock purchase plan | Healthcare benefits | Life insuranceMid-level Full TimeAtlanta, GA, United States20h ago
-
Software Engineer - X Data USD 125K-400KApache Flink | Apache Kafka | Apache Spark | BigQuery | Cause analysis401k retirement plan | Dental insurance | Discounts | Health insurance | Life insuranceMid-level Full TimePalo Alto, CA1d ago
-
Embedded Software Engineer Intern (Fall 2026) USD 108K-108KADC | Antenna Testing | C# | C++ | CANHousing stipend | Overtime pay | Paid sick time | Relocation supportEntry-level InternshipSouth San Francisco, California, USA1d ago
-
Site Reliability Engineer, Inference Infrastructure USD 165K-267KAmazon Web Services | C plus plus | Cloud platform | Distributed Systems | GPU401k matching | Co working Benefit | Company offsite | Education and learning stipend | Health and dental benefitsMid-level Full TimeToronto1d ago
-
Staff Software Engineer, Inference Infrastructure USD 160K-220KAmazon Web Services | Azure | C++ | Cloud platform | Distributed Systems401k | Annual company offsite | Arts and culture budget | Education and learning stipend | Health and dental benefitsSenior-level Full TimeSan Francisco1d ago
-
AI Engineer – Decision & Optimization Systems USD 80K-210KAMPL | C++ | CPLEX | Constraint Satisfaction | Go401k | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Engineer - Allocation and Packing Systems USD 80K-210KA/B | A/B Testing | Approximation algorithms | Assignment Algorithms | B testing401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Engineer – Routing & Network Optimization USD 80K-210KA Star | A Star Algorithm | AMPL | Bellman Ford Algorithm | CPLEX401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Software Engineer USD 80K-210KAWS Kinesis | Apache Airflow | Apache Kafka | C# | C++401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
Deployment Strategist - North America USD 150K-245KAI Agent | AI Agent Frameworks | API Integration | Agent Frameworks | Agent OrchestrationAnnual company offsite | Annual travel stipend | Co-working stipend | Professional development stipendMid-level Full TimeUnited States R1d ago
-
Applied AI Engineer USD 204K-352KBackend Development | Cloud infrastructure | Deep learning | Docker | Fine TuningAnnual membership | Centralized coffee stipend | Flexible PTO | Health and wellness stipend | Health, dental, and vision insuranceSenior-level Full TimeNew York, NY (HQ)1d ago
-
Senior Applied Scientist - Search USD 200K-200KData Science | Embedding | Fine Tuning | Hybrid search | Information Retrieval401k retirement | Annual leave | Equity package | Health coverage | Hybrid scheduleSenior-level Full TimeNew York City R1d ago
-
ML Research Engineer - Hardware Codesign USD 185K-455KC++ | CUDA | Floating point | Floating point numerics | Functional simulationHybrid work schedule | Relocation assistanceSenior-level Full TimeSan Francisco1d ago
-
Staff AI Engineer USD 140K-210KAPI Development | AWS | Access Management | Bedrock AgentCore | CI/CD401k | All-hands meetings | Dental insurance | Employee equity program | Health insuranceSenior-level Full TimeNew York City1d ago
-
Senior Quantum Embedded Engineer USD 142K-175K10G Ethernet | AMD Xilinx | AMD Xilinx toolchain | AMD/Xilinx SoC | BashHybrid work options | Remote work optionsSenior-level Full TimeNew Haven, CT1d ago
-
Associate Quantum Engineer USD 130K-184KCryogenic Systems | Data Analysis | Data acquisition | Instrument Control | Laboratory Instrument ControlInterdisciplinary team | Mentorship | State-of-the-art facilitiesMid-level Full TimeNew Haven, CT1d ago
-
Quantum Engineer (Physicist) USD 136K-187KCircuit-QED | Cryogenics | Data Analysis | Error correction | Low temperature physicsMid-level Full TimeNew Haven, CT1d ago
-
Senior Quantum Applications Engineer - QEC USD 120K-258KCUDA-Q | Decoder algorithms | Docker | End to End | End-to-End TestingSenior-level Full TimeNew Haven, CT1d ago
-
Staff AI engineer USD 170K-250KAI Evaluation | AWS | Agent Orchestration | Caching | Distributed SystemsFlexible working hours | Hybrid work culture | Unlimited time offSenior-level Full TimeSan Francisco1d ago