Model Serving Engineer
Tasks
- Build high-performance inference systems
- Configure autoscaling and GPU utilization
- Design inference serving platforms
- Ensure end-to-end observability
- Handle incident response for ML serving
- Implement request routing batching caching
- Manage capacity planning and performance tuning
- Operate production low latency services
Perks/Benefits
Skills/Tech-stack
Autoscaling | C++ | Caching | Capacity Planning | Distributed Systems | GPU Architecture | Go | Inference | Kubernetes | LLM | Metrics | Performance Engineering | Python | Rust | Structured Logging | TensorRT-LLM | Tracing | VLLM
Education
Roles
Related jobs
-
Senior AI Engineer – AI Platform USD 140K-160KAWS | Agentic AI | Alerting | Alignment | Distributed SystemsDaily meal specials | Dry cleaning | Free shuttle transportation | Gourmet cafeteria | Group fitness classesSenior-level Full TimeEnglewood Cliffs, NEW JERSEY, United States R3h ago
-
Software Engineer, Data Integration USD 165K-225KAWS | Automated testing | Azure | Cloud Computing | Data IngestionSenior-level Full TimeRemote - US Only R17h ago
-
Software Engineer - AI Developer Productivity USD 165K-330KAudit Logging | Authentication | Authorization | CI/CD | Context ManagementCompany facilitated 401k | Fertility and family building stipend | Flexible PTO | Medical dental vision insurance 100 percent | Paid parental leaveSenior-level Full TimeSan Francisco R17h ago
-
Telemetry Data Engineer USD 110K-128KAnomaly Detection | Azure Data | Azure Data Explorer | Data Engineering | Data Explorer401k retirement plan | Medical/Dental/VisionMid-level ContractRedmond, Washington, USA R17h ago
-
Senior AI Engineer USD 176K-258KAgentic Workflows | Agile | Cloud Architecture | Code review | DatabricksCell phone allowance | Comprehensive health coverage | Equity grants | Growth-focused environment | Home office setup allowanceSenior-level Full TimeRemote - US R17h ago
-
Senior AI Engineer - Grafana AI/ML | USA | Remote USD 127K-203KAWS | Agent Frameworks | Agent workflows | Artificial Intelligence | AzureAnnual leave | Company funded AI coding assistants usage budget | Equity | RSUs | Remote workSenior-level Full TimeUnited States (Remote) R18h ago
-
Automated testing | Build Automation | C++ | CI/CD | Firmware401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilityMid-level ContractDallas, Texas, USA R18h ago
-
AWS | AWS EC2 | Automation | Azure | Azure Virtual401k match | ESPP | Family leave programs | Flexible time away | Flexible-hybrid workSenior-level Full TimeSan Diego, CALIFORNIA, United States R18h ago
-
Sr. Analytics Engineer/ Data Scientist USD 205K-249KAirflow | Amazon Redshift | Anomaly Detection | BigQuery | CI/CD401k | Bi annual company offsites | Commuter stipend | Dental insurance | EquitySenior-level Full TimeSan Francisco, CA R19h ago
-
Senior Software Engineer - Cortex AI - FDE USD 200K-270KCloud Native | Cloud Native Architecture | Database Internals | Distributed State | Distributed SystemsSenior-level Full TimeUS-CA-Menlo Park R19h ago
-
AWS | Azure | CI/CD | Cloud platform | Containers401k matching | ESPP | Family leave programs | Flexible spending accounts | Flexible time away planSenior-level Full TimeSan Diego, CALIFORNIA, United States R19h ago
-
Staff AI Engineer USD 198K-260KAgentic Workflows | Containerization | Docker | Experiment design | Generative AI401k plan participation | Annual performance bonuses | Company equity grants | Disability insurance | Life insuranceSenior-level Full TimeNew York, New York, United States R19h ago
-
Senior AI Engineer USD 120K-160KAWS | Databricks | FastAPI | Flask | Langgraph401k Safe Harbor Plan | Background check required | Certification reimbursement | Free health insurance option | Generous PTOSenior-level Full TimeSchenectady, New York, United States, Remote R19h ago
-
Staff Embedded Security Engineer USD 276K-311KBootloader development | C# | C++ | Code review | CryptographyCompetitive equity package | Hybrid workSenior-level Full TimeSunnyvale R19h ago
-
Automated testing | C++ | CI/CD | Embedded Linux | Firmware401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilityMid-level ContractTampa, Florida, USA R20h ago
-
Automated testing | C++ | CI/CD | Firmware Build | Firmware Testing401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilityMid-level ContractAtlanta, Georgia, USA R20h ago
-
Data Engineer USD 75K-91KAccess Control | CI/CD | DBT | Data Modeling | Data Pipeline Monitoring401-k match | Flexible work schedule | Paid time off | Tuition reimbursement | Volunteer timeMid-level Full TimePlantation, Florida, United States R20h ago
-
Machine Learning Engineer, II - 3D Perception USD 153K-183K3D Perception | BEV Representations | Birds Eye View | C++ | Cloud processing100 percent paid medical dental and vision | 401k employer match | Accidental death and dismemberment insurance | Company holiday office closures | Life insuranceMid-level Full TimeRemote, U.S, Ann Arbor, MI, Fort … R21h ago
-
Software Engineer, AI & Security USD 140K-265KAI Security | AI model | Data Pipelines | Distributed Systems | Golang401k matching | Dental | Education stipend | Healthy lunches | Home office stipendMid-level Full TimeMountain View, CA R21h ago
-
Biomedical AI | Computer Vision | Data Pipelines | Deployment Pipelines | ML workflowsSenior-level Full TimeBoston (Onsite), New York (Onsite) Preferred, … R21h ago
-
Principal Software Engineer, Data USD 190K-210KAgentic coding | Amazon Kinesis | Apache Pulsar | Containerization | Data ModelingPTO | Remote workSenior-level Full TimeBoston, MA / Remote R21h ago
-
Principal Data Engineer USD 190K-210KAmazon Kinesis | Apache Pulsar | Containerization | Data Modeling | Event ProcessingEquity | PTO | Remote workSenior-level Full TimeBoston, MA / Remote R21h ago
-
AWS | AWS Batch | AWS Bedrock | AWS CDK | AWS IoTHealthcare | PTO | Paid Holidays | Phone stipend | Wellness stipendSenior-level Full TimeSan Carlos - Hybrid R21h ago
-
Autonomy | C plus plus | CPU GPU | CPU/GPU Optimization | Code review401k | Health insurance | Paid Holidays | Paid time off | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R22h ago
-
API Design | API Gateway | AWS Lambda | Amazon Aurora | Amazon DynamoDB401k matching | Accident insurance | Dental insurance | Education reimbursement | Health insuranceSenior-level Full TimeOffice Location or Remote - USA R22h ago