Product Manager - AI Inference & Model Serving
Austin, TX, United States
USD 160K-275K (estimate) Mid-level Full Time
Tasks
- Create PoC playbooks and sizing guides
- Define lifecycle for inference services
- Define performance outcome metrics and improvement plans
- Drive go to market pricing packaging and reference architectures
- Lead technical discovery with platform engineering teams
- Manage observability and reliability requirements
- Own product strategy and roadmap for AI inference and model serving
- Partner on system design trade offs for runtime GPU scheduling and serving topology
- Translate findings into prioritized requirements and architecture direction
Perks/Benefits
Skills/Tech-stack
AI Inference | Autoscaling | Cache Management | Cold Start | Cold Start Optimization | Continuous batching | Dedicated Endpoints | Disaggregated serving | DynamoDB | GPU scheduling | Inference Server | KV cache | KV-cache management | Model Serving | Multi model serving | Multi-model | Network Optimization | Observability | Performance Engineering | Prefill Decode | Prefill Decode Optimization | Reliability Engineering | Routing | SGLang | Serverless | Storage Optimization | TensorRT-LLM | Triton Inference | Triton Inference Server | VLLM | Workload placement
Education
N/A
Roles
Related jobs
-
Support Operations Manager, Analytics & Workforce USD 112K-161KAI Containment | Average Handle Time | Capacity Planning | Contact center | Contact center operationsHealth and wellness benefits | Hybrid work options | Paid time off | Remote workMid-level Full TimeRemote, US R19h ago
-
Technical Program Manager, MTIA Software USD 167K-230KAI Algorithms | AI Inference | AI Training | AI accelerator | BenchmarkingMid-level Full TimeMenlo Park, CA3d ago
-
AWS | Alerting | Black box monitoring | Black-box | CI/CDBackup childcare | Financial coaching | Mental health support | Mentoring | Onsite health and wellness centersSenior-level Full TimeJersey City, NJ, United States3d ago
-
Senior Product Manager, Data Platform (Remote) USD 162K-215KAI | Access Control | Business Intelligence | Cost Optimization | Data CatalogBusiness events | Remote work | Team gatherings | Together Weeks | Travel up to 5 days per quarterSenior-level Full TimeBoston, MA R3d ago
-
Senior Product Manager - Agent Integrations USD 192K-240KAI Agents | Backlog Management | BentoML | Customer discovery | Data AnalysisContinuous professional development | Inclusive community culture | Mental health benefits | Mentor/Buddy program | Stock equitySenior-level Full TimeNew York, New York, USA3d ago
-
Manager, Data Platform Operations USD 101K-131K.NET | AWS | AWS CloudFormation | Access Controls | Apache Airflow401k match | Company paid life insurance | Company-paid disability insurance | Dental insurance | Health insuranceMid-level Full TimeSaint Louis, Missouri, United States4d ago
-
Databricks Data Engineering Manager USD 151K-252KArtificial Intelligence | Business Intelligence | Data Architecture | Data Engineering | Data GovernanceMentorship | Professional development | Travel up to 25 percentMid-level Full TimeArlington/Rosslyn, Virginia, United States; Sacramento, California, …4d ago
-
AI Transformation Group Manager USD 176K-265KAkka | Apache Spark | Artificial Intelligence | Batch Processing | CI/CDSenior-level Full TimeLocation(s): Jersey City, New Jersey, United …4d ago
-
Head of AI Products USD 250K-300KArtificial Intelligence | Braintrust | Cost Optimization | Data Warehousing | EvaluationEquity compensation | In office Mondays and WednesdaysExecutive-level Full TimeSanta Clara4d ago
-
Senior Product Manager, Data Platform & Intelligence USD 139K-200KAmazon Redshift | BigQuery | Context injection | Cost Optimization | Data Governance401k match | Dental insurance | Fertility assistance | Fidelity Company Match Bonus After 7 Years | Flexible time offSenior-level Full TimeUS-Remote R4d ago
-
Engineering Manager, ML Performance USD 207K-301KAuto sharding | Benchmarking | CUDA | CUDA Performance | Compiler optimizationSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA5d ago
-
Manager, Enterprise Data Engineering USD 180K-247KADLS | Active Directory | Agile | Apache Spark | Automated testing401k matching | Adoption Assistance | Childcare tuition discounts | Company bonus | Fertility benefitsSenior-level Full TimeAnn Arbor, MI, United States5d ago
-
AIOps | Automation | Business Continuity | CI/CD | Capacity ManagementMid-level Full Time142019-NC-300 South Brevard, Charlotte, United States5d ago
-
Senior Product Manager - Unstructured Cloud Storage USD 140K-210KAgile methodology | Apache Spark | Azure | C# | Cloud Platform ArchitectureHybrid work modelSenior-level Full TimeIllinois, United States5d ago
-
Manager, Software Engineering – BI & Analytics USD 171K-252KAWS Aurora | AWS EMR | AWS Glue | AWS Lambda | AWS S3Mid-level Full TimeUSA - CA - 1200 Grand …5d ago
-
Datadog for Startups - Forward Deployed Engineering Lead USD 192K-240KAPM | AWS | Automation | Azure | CI/CD401k match | Dental benefits | Discounted employee stock purchase plan | Fitness reimbursements | Healthcare benefitsSenior-level Full TimeNew York, New York, USA6d ago
-
Senior Manager, Data Platform & Autonomy Infrastructure USD 225K-275KAutonomy | Data Engineering | Data Modeling | Data Pipelines | Data StorageDental insurance | Equity compensation | Health insurance | Paid time off | Performance bonusSenior-level Full TimeSouth San Francisco, California, USA6d ago
-
AI | Analytics | Automation | Cloud | Data ArchitectureHealthcare benefits | Hybrid work schedule | Paid time off | Retirement planMid-level Full TimeOhio - Columbus, Three Nationwide Plaza, … R6d ago
-
Auditability | Backlog Management | Billing | Compliance | Data ModelingMid-level Full TimeGM Global Technical Center - 7000 …6d ago
-
Best practices | Budget Management | Cross-Functional Collaboration | Cross-functional | Deliverable Management401k | Co working Benefit | Company offsite | Education and learning stipend | Health and dental benefitsSenior-level Full TimeCanada7d ago
-
Staff Software Engineer, AI/ML USD 186K-233KApache Iceberg | Capacity Planning | Cloud services | Competitive Analysis | Data ArchitectureConference reimbursement | Education reimbursement | Employee assistance program | Employee stock purchase program | Flexible time offSenior-level Full TimeSeattle7d ago
-
Data Science Senior Manager USD 244K-553KAWS | Agentic AI | Artificial Intelligence | Azure | CI/CDSenior-level Full TimeNewport Beach, CA, US, 926607d ago
-
Sr. Manager Data Science - Gen AI and Content Systems USD 118K-219KAWS | Azure | CI/CD | Containerization | Data Pipelines401k | Dental benefits | Flexible working hours | Health benefits | SabbaticalSenior-level Full TimeUSA - Raleigh, NC (RDU), United …7d ago
-
Engineering Manager, LLM Performance USD 224K-431KAPI Development | C++ | CUDA | Distributed Systems | GPU ArchitectureEquity | Health benefits | Hybrid workMid-level Full TimeUS, CA, Santa Clara, United States7d ago
-
Sr. Manager Data Science - Gen AI and Content Systems USD 118K-219KAWS | Automation | Azure | Benchmarking | CI/CD401(k) retirement benefits | Dental benefits | Flexible work hours | Health benefits | SabbaticalsSenior-level Full TimeUSA - Raleigh, NC (RDU), United …7d ago