Senior Software Engineer II, Inference
Tasks
- Apply graceful degradation
- Decompose work into milestones
- Define SLIs and SLOs
- Drive system architecture
- Execute rollback and traffic shift strategies
- Implement model serving optimizations
- Improve coding and testing standards
- Lead design reviews
- Lead multi service reliability improvements
- Mentor engineers
- Own post incident actions
- Plan capacity and autoscaling
- Quantify performance impact
- Review cross-team designs
Perks/Benefits
- 401k match
- Employee stock purchase program
- Flexible PTO
- Flexible spending account
- Health savings account
- Life insurance
- Medical/Dental/Vision insurance
- Paid parental leave
- Tuition reimbursement
- Tuition support
Skills/Tech-stack
Autoscaling | BF16 | C++ | CI/CD | CUDA | Caching | Capacity Planning | Distributed Systems | FP8 | GPU Computing | Go | Grafana | Incident Management | Inference Batching | KV cache | Kubernetes | Latency optimization | Microbatching | Mixed Precision | NCCL | NUMA | Networking | OpenTelemetry | Performance Engineering | Prometheus | Python | RDMA | Ray Serve | Reliability Engineering | SLI | SLO | Speculative decoding | Streaming token delivery | TensorRT-LLM | Throughput Optimization | Torchserve | Triton | VLLM
Education
N/A
Regions
Countries
States
Related jobs
-
Sr. Staff Software Engineer - Managed Tables USD 228K-314KAlgorithms | Automation | Capacity Planning | Data Structures | Database systemsSenior-level Full TimeMountain View, California6h ago
-
Forward Deployed Machine Learning Engineer USD 180K-300KAPI Design | Cloud Computing | Deep learning | Diffusion Models | Fine TuningIn-person collaboration days | Remote work flexibility | Travel cost coverageSenior-level Full TimeSan Francisco (USA) R8h ago
-
Senior Software Engineer, Knowledge Graph USD 196K-230KApache Flink | Apache Spark | Data Ingestion | Data Processing | DatabasesEmployee travel credits | Remote eligibleSenior-level Full TimeUnited States9h ago
-
Lead Data Engineer USD 115K-130KAgile | Apache Airflow | Azure Data | Azure Data Factory | Cloud Orchestration401k | Dental insurance | Medical insurance | Paid leave | Tuition reimbursementSenior-level Full TimeUniversal City, CALIFORNIA, United States10h ago
-
Cyber Data Engineer USD 140K-145KAWS | ArcSight | Bash | Cribl | DHCP401k match | Accrued PTO | Health/Dental/Vision | Life insurance | Long-term disabilitySenior-level Full TimeSpringfield, VA11h ago
-
Ansible | ArgoCD | CI/CD | Chef | Configuration ManagementSenior-level Full TimeNew York, NY, United States11h ago
-
Mechanical Engineer, Data Centers - Remote (U.S.) USD 150K-175KAir Cooled Systems | Air Filtration | Air handlers | Air quality | Air quality monitoringOccasional travel | Remote work optionsSenior-level Full TimeArlington, VA, United States R12h ago
-
Senior Engineer Embedded System Engineer USD 94K-125K8D methodology | A3 problem solving | Board Design | Bus Traces | Bus analyzerLife, accident, and disability insurance | Medical/Dental/Vision insurance | Paid sick leave | Paid vacation time | Tax-advantaged flexible spendingSenior-level Full TimeIrvine, CA, United States14h ago
-
A/B | A/B Testing | APIs | Airflow | B testingSenior-level Full TimeUnited States14h ago
-
Senior Systems Analyst – Robotic Algorithms and Control USD 134K-201KC# | C++ | CAD Tools | Classical control | Computer VisionSenior-level Full TimeSunnyvale, CA, United States14h ago
-
Full Stack Software Engineer - Robotics USD 125K-200KAWS | Datadog | Distributed Systems | Edge Computing | Grafana401k | Cell phone reimbursement | DC FSA | Employee assistance program | EquityMid-level Full TimeSan Francisco || Oakland, CA R15h ago
-
Senior Research Engineer, Voice + Speech USD 200K-400KData Pipelines | LLM | Language Processing | Machine Learning | Model EvaluationDaily meals snacks | Disability benefits | Fertility benefits | Life insurance | Medical/Dental/VisionSenior-level Full TimeNew York City15h ago
-
Staff Research Engineer, Voice + Speech USD 200K-400KConversational AI | Data Pipelines | Deep learning | Information Retrieval | LLM DeploymentDaily lunches and snacks | Disability benefits | Fertility and family building benefits | Life insurance | Medical/Dental/Vision insuranceSenior-level Full TimeNew York City15h ago
-
Senior-level Full TimeCosta Mesa, California, United States15h ago
-
Senior Research Engineer, Voice + Speech USD 200K-400KData Pipelines | Deep learning | Information Retrieval | LLM Deployment | Language ModelsDaily meals | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeSan Francisco15h ago
-
AI Platform Engineer, Training and Inference USD 150K-225KANN indexing | BF16 | DDP | Embeddings | FP8Career growth | Learning opportunitiesSenior-level Full TimeSan Francisco15h ago
-
API Gateway | AWS Glue | Access Control | Apache Airflow | Apache AtlasSenior-level Full TimeFAIRFAX, VA, United States16h ago
-
AI Developer - Model Creation & Full Stack USD 150K-175KAWS | Angular | Azure | CI/CD | D3.jsRemote work | USPS Public Trust Clearance eligibleMid-level Full TimeWork from home, VA, United States R16h ago
-
Software/Embedded Systems Engineer USD 135K-158KAgile Development | C++ | Change Control | DoD Systems | Embedded SoftwareTravel 25%Senior-level Full TimeArlington, VA, United States16h ago
-
Data/ML Scientist SME USD 105K-150KAWS GovCloud | Anomaly Detection | Apache Spark | Bayesian Causal Inference | Bayesian MethodsMid-level Full TimeFAIRFAX, VA, United States16h ago
-
AI Data Engineer USD 165K-225KAudit Columns | CI/CD | Cortex Analyst | Cortex Complete | DBT401k company match | Flexible paid time off | Learning and development | Medical benefits | Paid parental leaveSenior-level Full TimePhiladelphia, PA, United States16h ago
-
AWS | Agile | Azure | Cloud platform | DB2401k match | Disability insurance | Life insurance | Medical, dental, and vision insurance | Paid bench timeSenior-level Full TimeCincinnati, Ohio, United States16h ago
-
Lead AI Engineer USD 180K-280KCI/CD | ColBERT | Docker | Faiss | Fine Tuning401k match | Bonus | Childcare benefits | Dental insurance | Disability insuranceSenior-level Full TimeQuincy, MA, United States16h ago
-
AI Developer USD 140K-180KAPI Integration | Agile | Authentication | Azure | Azure FunctionsPaid Holidays | Paid time off | Tuition reimbursementEntry-level Full TimeDallas, Texas, United States17h ago
-
API Integration | AWS | AWS Glue | Batch Processing | Code reviewSenior-level Full TimeIndianapolis, IN, United States R17h ago