Senior Director, Inference Products and Optimizations
Tasks
- Architect model serving and optimization layer
- Collaborate with product management on priorities and dependencies
- Coordinate incident response and maintain customer SLAs
- Define execute product roadmap for inference products
- Design and evolve inference serving stack
- Ensure production health stability and on call rotation
- Implement quantization KV cache management and speculative decoding
- Lead disaggregated serving approaches
- Manage inference metrics to meet SLOs
- Operationalize benchmarking observability and auto tuning
- Optimize throughput latency and GPU utilization
- Recruit mentor and coach engineers
- Support open source inference engines
Perks/Benefits
- Conference reimbursement
- Education reimbursement
- Employee assistance program
- Employee stock purchase program
- Equity compensation
- Flexible time off
- Hybrid work
- LinkedIn Learning access
- Local Employee Meetups
- Training reimbursement
Skills/Tech-stack
AI workload | AI workload orchestration | AMD GPU | CUDA | Container Runtime | Container Runtime Security | Disaggregated serving | Distributed Systems | GPU Architecture | Hardware topology | KV cache | Kubernetes | LLM Inference | Microservices | NVIDIA GPU | NVLink | Observability | Quantization | Resource Management | Runtime Security | SGLang | SLO | Scheduling | Speculative decoding | TPOT | TTFT | VLLM | Workload Orchestration
Education
N/A
Related jobs
-
VP Software Engineering, Robotics & System USD 250K-320KAI machine learning | Cloud Computing | Cloud Integrated Hardware Software Systems | Cloud systems | Code review401k match | Dental insurance | Health insurance | Paid Medical Dental Vision Coverage | Pre-tax commuter benefitsExecutive-level Full TimeSan Francisco1d ago
-
Senior-level Full TimeBoston R1d ago
-
Technical Program Manager, Data Engineering BRL 150K-160KBatch Processing | Data Governance | Data Lakes | Data Modeling | Data QualityMid-level Full TimeArlington, VA - 1801 S Bell …2d ago
-
Director, AI Engineering and Delivery USD 202K-252KAmazon SageMaker | Amazon Web Services | Azure | Azure Machine Learning | CI/CDDental insurance | Employee assistance program | Employee resource groups | Flexible time off | Flexible work arrangementsExecutive-level Full TimeShakopee, MN, United States2d ago
-
Manager, Data Platform - AI Engineering USD 160K-192KAI Agents | AWS | Azure | CI/CD | Compliance401k match | Employer provided vehicle | Medical dental vision income protection | Mobility stipend | Paid leave for bereavement/jury duty/voting/militarySenior-level Full TimeCharlotte, North Carolina, United States2d ago
-
Senior-level Full TimePhoenix, AZ, United States2d ago
-
Observability Product Manager with AI | Dynatrace SaaS | Burlington, MA | Hybrid | Independent Visa Required USD 150K-180KAIOps | Anomaly Detection | Artificial Intelligence | Automation | CI/CDConfidentiality | Hybrid work schedule | Long-term projectMid-level Contract Full TimeBurlington, MA, United States2d ago
-
API Integration | Agile | Artificial Intelligence | Automated testing | CI/CDMid-level Full TimeCupertino2d ago
-
Senior Director, Data Architecture (Next Gen) USD 217K-245KAKS | Apache Flink | Apache Kafka | Azure Cosmos | Azure Cosmos DBSenior-level Full TimeUS - GA - Atlanta, United …3d ago
-
Director, Product Management - Enterprise AI/ML USD 209K-286KAI Risk | AI risk management | AWS | Agentic AI | Apache SparkExecutive-level Full TimeMcLean, VA, United States3d ago
-
Director, Data Architecture - Las Vegas, NV USD 140K-170KAWS | Cloud Resource Management | Data Contracts | Data Governance | Data Lifecycle Management401k Company Contributions | Disability coverage | Discretionary bonus opportunities | Health, dental and vision coverage | Identity protectionSenior-level Full TimeLas Vegas, NV, US, 89113 R3d ago
-
Lead Principal Product Manager - Robotics USD 119K-264KAI strategy | Autonomy | Business case | Business case development | Case Development401k match | Adoption Assistance | Commuter benefits | Employee stock purchase plan | Flexible spending accountsSenior-level Full TimeUnited States3d ago
-
A Bn Testing | A/B | A/B Testing | API Integration | API firstBackup childcare | Financial coaching | Health care coverage | Mental health support | Retirement savings planExecutive-level Full TimeColumbus, OH, United States3d ago
-
Manager II, Data Platform USD 154K-234KAWS | AWS DMS | AWS Fargate | Amazon DynamoDB | Amazon ECSComprehensive health insurance | Flexible remote work | Parental leave | Professional development stipendMid-level Full TimeRemote - US R3d ago
-
Senior Manager, AI Agent / ML Engineering USD 99K-234KAccess Control | AgentOps | Auditability | Autogen | Cost OptimizationSenior-level Full TimeNashville, TN, United States3d ago
-
Enrollment Manager CT USD 55K-58KData Analysis | DocuSign | Employee Coaching | Microsoft Excel | Process ImprovementBenefits | Competitive wages | Flexible paid time off | Professional growth opportunities | Supportive work environmentMid-level Full TimeSouthington, Connecticut, United States4d ago
-
Team Lead, AI Software Development USD 123K-235KAI Coding Agents | AI coding | Agent architecture | Agile | Artificial Intelligence401k employer match | All-hands meetings | Business casual dress code | English fluency | Flexible PTOSenior-level Full TimeAustin, United States4d ago
-
Data Pipelines | Data Processing | Debugging | Distributed Systems | Fine TuningSenior-level Full TimePittsburgh, PA, USA4d ago
-
AI Governance | Agent systems | Google Cloud | Language Models | Large Language ModelsHealth insurance | Paid time off | Parental leave | Professional development | Retirement planSenior-level Full TimeChicago, IL, USA; Austin, TX, USA4d ago
-
Adaptive Learning Loops | Adaptive learning | Agent systems | Agentic Workflows | Cloud DatabasesSenior-level Full TimeSunnyvale, CA, USA4d ago
-
Sr Manager, Software Data Engineering USD 188K-252KAngular | Apache Spark | CI/CD | Cloud Computing | DatabricksSenior-level Full TimeUSA - CA - Studio Lot …4d ago
-
APIs | Automation | Ceph | Consistency models | ContainersCertification assistance | Flexible work arrangements | Hybrid work environment | MentorshipSenior-level Full TimeWA Remote Zone 1, United States R4d ago
-
Adversarial Testing | Agent Orchestration | Agent systems | Artificial Intelligence | Caching401k matching | Basic life insurance | Dental insurance | Long-term disability | Medical insuranceExecutive-level Full TimeSan Jose, California, United States of …4d ago
-
Manager, Data Engineering USD 140K-185KAzure Data | Azure Data Bricks | Boome | CI/CD | DBT401k match | Dental insurance | Dependent care spending account | EAP | Flexible Friday early log offMid-level Full TimeRemote (United States) R4d ago
-
A/B | A/B Testing | API Integration | AWS | Agent systemsCommuter stipend | Comprehensive health, dental and vision coverage | Generous PTO | Learning and development stipend | Retirement benefitsMid-level Full TimeSan Francisco, CA; New York, NY4d ago