Senior Director, Inference Products and Optimizations
Tasks
- Collaborate with product management and stakeholders
- Define and execute product roadmap
- Design inference serving stack
- Ensure production health stability and on call operations
- Implement benchmarking observability and auto tuning
- Lead inference product team
- Maintain service level objectives
- Manage quantization KV cache and speculative decoding
- Optimize throughput latency and GPU utilization
- Recruit mentor and coach engineers
Perks/Benefits
Skills/Tech-stack
AI workload | AI workload orchestration | AMD | Benchmarking | Call Management | Container runtimes | Distributed Systems | GPU Architecture | KV cache | Kubernetes | LLM Inference | LLMD | Microservices | Model Serving | NVIDIA | NVLink | Observability | On-Call | On-call Management | Quantization | Resource Management | SGLang | SLOs | Scheduling | Security contexts | Speculative decoding | VLLM | Workload Orchestration
Education
N/A
Regions
Countries
States
Related jobs
-
Senior Machine Learning Engineering Manager USD 295K-345KCloud services | Computer Vision | Data Pipelines | Data Quality | Distributed SystemsEquity compensation | Health benefitsSenior-level Full TimeSan Mateo, CA, United States R1d ago
-
Engineering Manager, Online Storage - SF USD 250K-300KAWS | Amazon Aurora | Amazon RDS | Amazon S3 | Disaster RecoveryEmployee assistance program | Flexible PTO | Home office stipend | Hybrid work | Monthly commuter benefitSenior-level Full TimeSan Francisco R1d ago
-
Senior Engineering Manager, Enterprise AI USD 184K-230KAWS | Apache Airflow | Apache Hive | Apache Impala | Apache SparkCareer development | Employee resource groups | Flexible WFH | Generous PTO | Paid volunteer timeSenior-level Full TimeUS-Connecticut-Remote, United States R2d ago
-
Director - Data Engineering USD 215K-275KAWS | Amazon Web Services | Azure | Bash | Container OrchestrationFlexible time off | Healthcare | Hybrid work | Leave benefits | Retirement benefitsExecutive-level Full TimeSF4-San Francisco - 400 Howard Street, … R3d ago
-
API Design | CI/CD | Continuous Delivery | Continuous integration | DORA MetricsMid-level Full TimeSan Jose, United States R3d ago
-
Sr. Manager, Machine Learning USD 160K-317KAgile | Cost Optimization | Deep learning | Evaluation | Inference OptimizationSenior-level Full TimeSan Francisco, United States R4d ago
-
Director, SteadyMD Workforce Optimization USD 125K-145KAnalytics | Aspect | Automation | Capacity Planning | Contact center401k | Dental insurance | Medical insurance | Paid time off | Vision insuranceExecutive-level Full TimeRemote Employee, United States R4d ago
-
Director Data Science - Remote or Hybrid in MN or DC USD 134K-230KAI Governance | Agent Orchestration | Agentic AI | Bias Assessment | Context engineering401k contribution | Comprehensive benefits package | Equity/Stock Purchase Plan | Incentive and recognition programs | Remote flexibilityExecutive-level Full TimeEden Prairie, Minnesota, UNITED STATES; REMOTE R7d ago
-
Agentic AI | Azure | Cloud Native | Cloud platform | Compliance401k contribution | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programsExecutive-level Full TimePrimary location: Eden Prairie, Minnesota R8d ago
-
Director, AI, Data and Developer Enablement USD 165K-275KAccess Controls | Apache Kafka | Apache Spark | CI/CD | Change ManagementDevelopment programs for advancement and career growth | Paid education assistance | Paid parental leave | Scheduling flexibility | Team member discountExecutive-level Full Time985 - Fred Meijer Office, United … R8d ago
-
Associate Director, Data Engineering (Remote) USD 130K-140KAutomated testing | CI/CD | Continuous Delivery | Continuous integration | Data Governance401k company match | Career growth opportunities | Company holidays | Generous PTO | Medical/Dental/Vision insuranceMid-level Full TimePortland R8d ago
-
Director, Enterprise Data Engineering & Analytics USD 139K-277KAWS | Amazon Kinesis | Amazon Redshift | Apache Spark | Azure401k savings | Employee assistance programs | Health and welfare benefits | Hybrid work flexibility | Inclusive cultureExecutive-level Full TimeNew York - 1166, United States R10d ago
-
AWS | Access Management | Agent Development | BigQuery | CI/CD401k | Disability insurance | EAP Resources | Holidays | Life insuranceMid-level Full TimeSeattle WA, United States R11d ago
-
Director, Data Architecture & Engineering USD 162K-244KAPI Design | Agile | Artificial Intelligence | Automation | AzureSenior-level Full TimeRemote - US R14d ago
-
VP, AI Engineering & Agent Platforms USD 280K-340KAI infrastructure | AWS | Agent Orchestration | Cloud Native | Distributed Systems401k match | Company holidays | Dental insurance | Disability insurance | Employee assistance programExecutive-level Full TimeUnited States R15d ago
-
Executive-level Full TimeRemote, United States R16d ago
-
Program Director - Mainframe Optimization & Delivery- (ISV migrations and software swaps) USD 155K-210KAsana | Automation | Cost Optimization | Jira | MS Project401k company match | Dental & vision coverage | Disability coverage | Education reimbursement | Fitness center accessExecutive-level Full TimeRemote - United States R17d ago
-
Director, Analytics Engineering USD 240K-270KAlerting | Automated testing | CI/CD | DBT | Data GovernanceChildcare subsidy | Dell discount | Education discount | Employee assistance program | GympassExecutive-level Full TimeUnited States - Remote R17d ago
-
Director AI Engineering Platform USD 217K-319KAI Engineering | AI Observability | AI gateway | AI-assisted coding | AWSCasual work environment | Continuous improvement | Great Place to Work culture | Professional development | Work from homeExecutive-level Full TimeSilver Triangle Building, United States R21d ago
-
Machine Learning Engineering Manager, Critical Harms USD 295K-345KAutomation | Data Analysis | Distributed Systems | Engineering Management | ExperimentationEquity compensation | Health benefits | Onsite work schedule guidanceMid-level Full TimeSan Mateo, CA, United States R23d ago
-
Senior Engineering Manager, AI Quality & Governance USD 210K-230KAI Evaluation | AI Observability | AI RMF | AWS | Access Controls401k employer match | Career advancement | Employee recognition | Health, dental, vision insurance | LinkedIn Learning PlatformSenior-level Full TimeRemote US R24d ago
-
AWS | Agile | Cloud Computing | Data platform | Distributed SystemsRemote eligibleSenior-level Full TimeSan Francisco, CA, United States R28d ago
-
Distinguished Machine Learning Engineer - Safety USD 399K-457KComputer Vision | Data Architecture | Data Processing | Distributed Systems | Language ModelsEquity compensation | Onsite work schedule | Workplace inclusion cultureSenior-level Full TimeSan Mateo, CA, United States R29d ago
-
Sr Manager AI/ML Engineering USD 148K-255KAWS | Airflow | Amazon SageMaker | Auditability | Azure401k contribution | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programs | Remote work within United StatesSenior-level Full TimePrimary location: Eden Prairie, MN R29d ago
-
Director, Data Platform USD 170K-225KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageHybrid work | Remote 2 days per week | Supportive collaborative environment | Travel to client sitesExecutive-level Full TimeAtlanta; Boston; Charlotte; Chicago; Dallas; Los … R29d ago