Product Manager - AI Inference & Model Serving
Austin, TX, United States
USD 160K-275K (estimate) Mid-level Full Time
Tasks
- Create PoC playbooks and sizing guides
- Define lifecycle for inference services
- Define performance outcome metrics and improvement plans
- Drive go to market pricing packaging and reference architectures
- Lead technical discovery with platform engineering teams
- Manage observability and reliability requirements
- Own product strategy and roadmap for AI inference and model serving
- Partner on system design trade offs for runtime GPU scheduling and serving topology
- Translate findings into prioritized requirements and architecture direction
Perks/Benefits
Skills/Tech-stack
AI Inference | Autoscaling | Cache Management | Cold Start | Cold Start Optimization | Continuous batching | Dedicated Endpoints | Disaggregated serving | DynamoDB | GPU scheduling | Inference Server | KV cache | KV-cache management | Model Serving | Multi model serving | Multi-model | Network Optimization | Observability | Performance Engineering | Prefill Decode | Prefill Decode Optimization | Reliability Engineering | Routing | SGLang | Serverless | Storage Optimization | TensorRT-LLM | Triton Inference | Triton Inference Server | VLLM | Workload placement
Education
N/A
Roles
Related jobs
-
C# | C++ | Cloud platform | Code review | Distributed SystemsSenior-level Full TimeSeattle, WA, USA5h ago
-
Senior Manager, Data Engineering USD 240K-310KAWS | Amazon Redshift | Apache Airflow | CI/CD | CloudFormationCasual environment | Company retreats | Great benefits | Wellness programSenior-level Full TimeNew York, New York19h ago
-
Product Manager - AI/ML Solutions USD 158K-232KAI/ML | AI/ML Governance | Cross-functional | Cross-functional leadership | Data analyticsBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersMid-level Full TimeNew York, NY, United States22h ago
-
Manager, Data Engineering-IT USD 115K-204KAccess Controls | Apache Spark | Automation | Azure Data | Azure Data FactorySenior-level Full TimeUS-Kansas-Wichita1d ago
-
Product Manager – MDR Data & AI Solutions USD 102K-175KAPI Gateway | API Integration | AWS | AWS Glue | AWS Lake FormationSenior-level Full TimeUS121 NJ Raritan - 1000 Highway …1d ago
-
Manager, AI Engineering USD 122K-152KAI Act | AI RMF | Agent Frameworks | Bias Testing | Data Lineage401k match | Business travel coverage | Dental insurance | Disability insurance | Employee assistance programMid-level Full TimePrinceton, New Jersey, United States; San … R1d ago
-
Senior Lead AI Software Engineer USD 216K-324KAgentic Workflows | Alerting | Architectural patterns | Coding assistants | Cost OptimizationSenior-level Full TimeBoston, MA2d ago
-
Agentic Architectures | Agile | CI/CD | Cloud infrastructure | Continuous Delivery401k | Employee discount | Health benefits | Long-term disability | Paid national holidaysSenior-level Full Time7000 Target Pkwy N,NCD-0375 Brooklyn Park,MN …2d ago
-
API Design | API Testing | C++ | Caching | CheckpointingSenior-level Full TimeSan Francisco2d ago
-
Manager, Data Engineering USD 205K-261KAWS | AWS Lambda | Airflow | Amazon Kinesis | Amazon RedshiftCompany retreats | Wellness programMid-level Full TimeNew York, New York2d ago
-
Senior AI Engineering USD 150K-275KADLS | AI Search | Agentic Workflows | Azure AI | Azure AI SearchSenior-level Full TimePasadena, CA, United States3d ago
-
Access Management | BigQuery | CI/CD | Cloud Functions | Cloud RunOn-site collaborationSenior-level Full TimeAustin, TX, United States3d ago
-
Lead Machine Learning Engineer (Manager IC) USD 179K-225KAI Governance | AWS | Agentic AI | Azure | BedrockSenior-level Full TimeCambridge, MA, United States3d ago
-
AVP, AI Engineering & Delivery USD 156K-290KAIOps | API Integration | Artificial Intelligence | Automation | DeploymentExecutive-level Full TimeEdina, MN 55435, United States3d ago
-
Sr Robotics Systems Engineering Manager USD 137K-235K21 CFR part 820 | Accelerated life testing | Actuation | CFR Part 820 | Cause analysisSenior-level Full TimeUS362 MA Boston - 501 Boylston …3d ago
-
Manager, Data Engineering USD 96K-135KAWS Athena | AWS Glue | AWS S3 | Batch Processing | CI/CDHybrid work | Visa sponsorshipMid-level Full TimeUSA - 2 West Liberty Boulevard, …3d ago
-
Sr Manager, Site Reliability (SASE) USD 182K-294KAWS | Anomaly Detection | Azure | CI/CD | Canary DeploymentSenior-level Full TimeSanta Clara, CA3d ago
-
Customer Fulfillment Manager USD 140K-192KAccount Management | CRM | Customer escalation | Customer escalation management | CybersecurityEmployee benefits | Hybrid collaboration | Remote workMid-level Full TimeCalifornia, US3d ago
-
Director, Applied AI Product Manager USD 127K-250KAI Governance | APIs | Access Control | Agentic architecture | Artificial IntelligenceFlexible resources tools wellbeing programs | Paid leave | Paid volunteer timeExecutive-level Full TimeNew York, NY, United States6d ago
-
Sr. Manager – Data & AI Support Engineering USD 192K-264KAWS | Agentic AI | Apache Spark | Azure | Big DataSenior-level Full TimeTexas6d ago
-
Data Platform Lead USD 210KAWS | Agentic Workflows | BigQuery | CI/CD | DBT401k | Company paid sustainability subscription | Flex PTO | Medical, dental, vision plans | Paid parental leaveSenior-level Full TimeRemote (US) R6d ago
-
Manager, Partner AI Deployment Engineering - AWS USD 251K-335KAWS | Cloud Computing | Evaluation | Generative AI | JavaScriptHybrid work model | Relocation assistanceSenior-level Full TimeSan Francisco7d ago
-
Marketing Analytics Manager USD 241K-247KA/B | A/B Testing | B testing | Data Visualization | Data pipeline401k match | Accidental insurance | Dental insurance | Fitness reimbursement | Flexible PTOMid-level Full TimeNew York, NY7d ago
-
Technical Lead – Software Engineering (AI/Agents Focus) USD 128K-185KAI Agents | API Design | API Gateway | AWS Lambda | AgileClient onsite required | Hybrid workSenior-level Full TimeUSA7d ago
-
Manager, Machine Learning Engineer USD 139K-250KAutomation | Compliance | Data Pipelines | Data Quality | Deep learningCoaching and mentorship opportunities | Hybrid work modelMid-level Full TimeMalvern, PA, United States7d ago