Senior Software Engineer I, Inference
Tasks
- Decompose work into milestones
- Define SLIs/SLOs
- Drive system architecture
- Execute rollback and traffic shift strategies
- Implement graceful degradation
- Implement inference performance optimizations
- Improve coding and testing standards
- Improve reliability through post incident actions
- Lead design reviews
- Mentor engineers
- Own multi service area
- Plan capacity for incidents
- Quantify optimization impact
- Review cross-team designs
- Tune autoscaling policies
Perks/Benefits
- 401k match
- Dental insurance
- Disability insurance
- Employee stock purchase program
- Flexible PTO
- Flexible childcare support
- Flexible spending account
- Health savings account
- Life insurance
- Long-term disability
- Medical insurance
- Mental wellness benefits
- Paid parental leave
- Short-term disability
- Tuition reimbursement
- Vision insurance
Skills/Tech-stack
Adaptive scheduling | Autoscaling | BF16 | C++ | CI/CD | CUDA | Caching | Cost Per | Cost Per Token Analytics | Cost per token | Distributed Systems | FP8 | GPU Interconnect | GPU Resource Isolation | Go | Graceful Degradation | Grafana | Inference Batching | KV cache | Kubernetes | Kv cache reuse | Microbatch Scheduling | Mixed Precision | NCCL | NUMA | Network Performance | OpenTelemetry | Prometheus | Python | RDMA | Ray Serve | Request Routing | Resource Isolation | Rollback | SHARP | SLI | SLO | Service reliability | Speculative decoding | Streaming token delivery | Tail Latency | TensorRT-LLM | Token analytics | Torchserve | Traffic shifting | Triton | VLLM
Education
Regions
Countries
States
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R21d ago
-
Senior Performance Co-Design Engineer, LLM Serving USD 174K-252KC++ | Co-design | Computer Architecture | Deep learning | Deep learning inferenceSenior-level Full TimeSunnyvale, CA, USA1h ago
-
Automated testing | C# | C++ | CI/CD | Continuous DeploymentMid-level Full TimeMountain View, CA, USA1h ago
-
Staff Software Engineer, Storage Intelligence Platform USD 207K-300KC++ | Data Structures | Data Structures and Algorithms | Distributed Systems | Large-scaleSenior-level Full TimeSeattle, WA, USA1h ago
-
C++ | Data Processing | Debugging | Generative AI | Google CloudHealth insurance | Learning and development opportunities | Paid time off | Relocation assistance | Retirement planSenior-level Full TimeMountain View, CA, USA1h ago
-
C++ | Data Processing | Debugging | Language Models | Language ProcessingSenior-level Full TimeMountain View, CA, USA1h ago
-
Senior Software Engineer, AI/ML GenAI, Google Workspace USD 174K-252KAlgorithms | C++ | Data Processing | Data Structures | DebuggingSenior-level Full TimeSunnyvale, CA, USA1h ago
-
Principal Engineer, Data Center Sustainability USD 307K-427KC States | Carbon Accounting | Co-design | Computer Architecture | DVFSSenior-level Full TimeSunnyvale, CA, USA1h ago
-
Senior Software Engineer, Google Cloud Storage USD 174K-252KAlgorithms | C# | C++ | Cloud Computing | Data IntegritySenior-level Full TimeNew York, NY, USA1h ago
-
Senior Performance Co-design Engineer, LLM Training USD 174K-252KC++ | Co-design | Computer Architecture | Data parallelism | Distributed TrainingSenior-level Full TimeSunnyvale, CA, USA1h ago
-
Alerting | Automated remediation | Cause analysis | Chaos Engineering | Distributed StorageSenior-level Full TimeCambridge, MA, USA1h ago
-
Data Engineer - Emergency Preparedness and Response USD 117K-151KAzure | Azure Data | Azure Data Factory | Azure Databricks | Big DataEmergency response availability | Health safety wellbeing focus | Hybrid work scheduleMid-level Full TimeDetroit, MI, US5h ago
-
Software Engineer, Embedded Agentic AI USD 170K-210KAgent Orchestration | Agent systems | C# | C++ | Cloud PlatformsDisability benefits | Equity awards | Health insurance | Life insurance | Paid time offEntry-level Full TimeSan Jose, California8h ago
-
Staff Engineer, GenAI Tooling & DevEx (R5469) USD 121K-197K*nix | AWS | Access Management | Active Directory | Azure GovSenior-level Full TimeWashington, DC10h ago
-
Staff Machine Learning Engineer, Personalization USD 152K-277KAirflow | Apache Spark | BigQuery | Contrastive Learning | Deep learning401k match | Disability insurance | Electric Car Charging Station | Employee assistance program | Flexible spending accountSenior-level Full TimeMountain View, USA12h ago
-
Senior Applied AI Engineer USD 205K-238KAgentic Methodologies | Agentic Workflows | Deep learning | Fine Tuning | Generative AIFamily planning support | Flexible time off | Lifestyle stipend | Mental health support | Paid parental leaveSenior-level Full TimeSan Francisco, CA R12h ago
-
Aerospace Algorithms Engineer USD 90K-210KData Analysis | Mathematical Modeling | Physics modeling | Python | Simulation401k match | Dental insurance | Life insurance | Medical insurance | Paid HolidaysMid-level Full TimeCambridge, MA, Arlington, VA, Seattle, WA12h ago
-
Senior-level Full TimeCupertino12h ago
-
2027 Technology, Data, AI & Ventures Summer Internship Program - AI Engineer (MLOps) Intern USD 60K-70KAPIs | Automated testing | CI/CD | DevOps | Generative AICapstone project | Hybrid work | Networking opportunitiesEntry-level InternshipNew York, NY, US12h ago
-
AWS Glue | Amazon Redshift | Amazon Web Services | Cloud platform | DBTAdoption Assistance | Hybrid schedule | Leave programs | Student loan repayment programEntry-level InternshipNew York, NY, US12h ago
-
Mid-level Full TimeSan Diego12h ago
-
Senior-level Full TimeSan Diego12h ago
-
Application Software Engineer, Applied AI USD 125K-200K.NET | A/B | A/B Testing | AI Observability | Agent Orchestration401k plan | Company shuttles | Employee stock purchase plan | Life insurance | Long-term disability insuranceSenior-level Full TimeRedmond, WA13h ago
-
Application Software Engineer, Applied AI USD 160K-225K.NET | A/B | A/B Testing | AI Observability | Agent OrchestrationBackground check required | Extended hours weekends as needed | Onsite work requiredSenior-level Full TimeCape Canaveral, FL13h ago
-
Application Software Engineer, Applied AI USD 125K-160K.NET | A/B | A/B Testing | Agent Orchestration | Agentic Systems401k retirement plan | Life insurance | Long-term disability insurance | Medical/Dental/Vision insurance | Paid HolidaysSenior-level Full TimeVandenberg, CA13h ago