Senior Director, Inference Products and Optimizations
Tasks
- Architect model serving and optimization layer
- Collaborate with product management on priorities and dependencies
- Coordinate incident response and maintain customer SLAs
- Define execute product roadmap for inference products
- Design and evolve inference serving stack
- Ensure production health stability and on call rotation
- Implement quantization KV cache management and speculative decoding
- Lead disaggregated serving approaches
- Manage inference metrics to meet SLOs
- Operationalize benchmarking observability and auto tuning
- Optimize throughput latency and GPU utilization
- Recruit mentor and coach engineers
- Support open source inference engines
Perks/Benefits
- Conference reimbursement
- Education reimbursement
- Employee assistance program
- Employee stock purchase program
- Equity compensation
- Flexible time off
- Hybrid work
- LinkedIn Learning access
- Local Employee Meetups
- Training reimbursement
Skills/Tech-stack
AI workload | AI workload orchestration | AMD GPU | CUDA | Container Runtime | Container Runtime Security | Disaggregated serving | Distributed Systems | GPU Architecture | Hardware topology | KV cache | Kubernetes | LLM Inference | Microservices | NVIDIA GPU | NVLink | Observability | Quantization | Resource Management | Runtime Security | SGLang | SLO | Scheduling | Speculative decoding | TPOT | TTFT | VLLM | Workload Orchestration
Education
N/A
Related jobs
-
Engineering Manager - Embedded DevSecOps Enablement USD 138K-207KAI-powered development | AI-powered development tools | AWS | Automated testing | CI/CDAdoption Assistance | Charitable contribution match | Discount programs | Employee assistance programs | Employee purchase planSenior-level Full TimeUrbandale,Iowa,United States16h ago
-
Senior Director, Inference Products and Optimizations USD 274K-343KAI workload | AI workload orchestration | AMD | Benchmarking | Call ManagementEmployee assistance program | Flexible time off | LinkedIn LearningSenior-level Full TimeSan Francisco R17h ago
-
Director, Engineering - Cloud Observability USD 296K-370KAI-powered Tooling | AWS | Azure | Cloud Computing | Cloud Economics401k match | Career pathing | Community guilds | Continuous professional development | Dental benefitsExecutive-level Full TimeBoston, Massachusetts, USA; New York, New …17h ago
-
Senior Machine Learning Engineering Manager USD 295K-345KCloud services | Computer Vision | Data Pipelines | Data Quality | Distributed SystemsEquity compensation | Health benefitsSenior-level Full TimeSan Mateo, CA, United States R1d ago
-
Director of Data Platform Engineering USD 180K-220KAccess Control | Amazon Redshift | Apache Airflow | Apache Flink | Apache KafkaComprehensive health coverage | Remote-first workplace | Unlimited PTOExecutive-level Full TimeRemote - United States R1d ago
-
Engineering Manager, Online Storage - SF USD 250K-300KAWS | Amazon Aurora | Amazon RDS | Amazon S3 | Disaster RecoveryEmployee assistance program | Flexible PTO | Home office stipend | Hybrid work | Monthly commuter benefitSenior-level Full TimeSan Francisco R1d ago
-
Head of AI and Data Platform USD 172K-344KA/B | A/B Testing | AWS | Artificial Intelligence | Automated retrainingEducation benefit | Free medical coverage | Retirement savings plan | Student debt program | Tuition reimbursementExecutive-level Full TimeUnited States - Alameda : 2901 …1d ago
-
Applied AI ML Director USD 150K-210KComputer Vision | DAG pipelines | DeepSpeed | Distributed Training | FP16Backup childcare | Financial coaching | Health insurance | Mental health support | On-site health and wellness centersExecutive-level Full TimePalo Alto, CA, United States1d ago
-
Director, BTS Data Platforms USD 172K-336KAWS | Analytics | Artificial Intelligence | Budget Management | Cost Optimization401k | Dental insurance | Health insurance | Paid time off | Vision insuranceExecutive-level Full TimeNorth Chicago, IL, United States1d ago
-
Customer Success | Data Engineering | Distributed Systems | Presales | Program ManagementHybrid work | Travel up to 30%Senior-level Full TimeRemote - California; Remote - Colorado; … R1d ago
-
Analytics | Business Intelligence | Cause analysis | Dashboarding | Data AnalysisMid-level Full TimeVienna, VA, United States1d ago
-
Cost Management | Cross-Functional Collaboration | Cross-functional | Functional collaboration | MetricsEmployee assistance programs | Health benefits | Paid time off | Recovery BenefitsSenior-level Full TimeCosta Mesa, California, United States1d ago
-
Distributed Systems | Fault Tolerance | GPU Computing | JAX | Low LatencyBonus program | Company benefits program | Equity incentive planSenior-level Full TimeMountain View, California2d ago
-
AWS | AWS Cloud | Artificial Intelligence | Azure | C#Senior-level Full TimeSan Jose, CA, United States2d ago
-
Manager, Data Engineering USD 80K-96KAWS | AWS Glue | Amazon Athena | Batch Processing | CI/CDVisa sponsorshipMid-level Full TimeUSA - 2 West Liberty Boulevard, …2d ago
-
Staff Engineer - Data Engineering USD 145K-223KAWS | AWS SageMaker | Agile | Azure | Business Intelligence401k retirement plan | Flexible work/hybrid | Healthcare coverage | Paid parental leave | Paid time offSenior-level Full TimeChicago, United States2d ago
-
Product Manager, HPC/AI Software USD 111K-211KAPI Design | Artificial Intelligence | Cloud Computing | Cluster management | Cross-Functional CollaborationCareer growth opportunities | Health and wellbeing benefits | Hybrid work schedule | Inclusion and flexibility in work and personal needs | Professional development programsMid-level Full TimeSpring, Texas, United States of America2d ago
-
Senior Engineering Manager, Enterprise AI USD 184K-230KAWS | Apache Airflow | Apache Hive | Apache Impala | Apache SparkCareer development | Employee resource groups | Flexible WFH | Generous PTO | Paid volunteer timeSenior-level Full TimeUS-Connecticut-Remote, United States R2d ago
-
Head of AI & Agentic Platform Engineering USD 270K-500KA B Rollback | A/B | A/B Testing | Access Control | Anomaly DetectionHybrid workExecutive-level Full TimeUSA - NY - Headquarters, United …2d ago
-
Product Manager, HPC/AI Software USD 111K-211KAI | API Design | Acceptance criteria | Cloud infrastructure | Cluster managementCareer Development Programs | Health and wellbeing benefits | Hybrid work scheduleMid-level Full TimeSpring, Texas, United States of America2d ago
-
APIs | Authentication | Authorization | Data Governance | Data IngestionCaregiver support | Flexible PTO | HSA match | Health benefits | Home office stipendSenior-level Full TimeRemote - USA R2d ago
-
Senior Product Manager – Observability Data Platform USD 192K-240KCloud Computing | Customer discovery | Distributed Systems | Go-to-market | Infrastructure architectureCommunity guilds | Employee stock purchase plan | Hybrid work | Inclusion talks | Mentor and buddy programSenior-level Full TimeNew York, New York, USA2d ago
-
Manager, Data Platform Engineering USD 160K-201KAmazon Redshift | Batch Processing | Cost Optimization | Data Modeling | Data WarehousingMid-level Full TimeBoston, Massachusetts, United States R2d ago
-
Staff Product Manager, Observability Data Platforms USD 244K-305KArchitecture | Cost Optimization | Cost to serve | Data platform | Market AnalysisCareer pathing | Community guilds | Employee stock purchase plan | Employee wellness benefits | Fitness reimbursementsSenior-level Full TimeNew York, New York, USA2d ago
-
Senior Engineering Manager, Data Engineering USD 193K-253KAWS | Access Control | Amazon Redshift | Apache Airflow | CI/CD401k retirement savings plan | Flexible time off | Health, dental, and vision insurance | Mental health support | Parental leaveSenior-level Full TimeRemote, USA R2d ago