Engineering Manager, Inference ML Runtime
Sunnyvale CA or Toronto Canada
USD 180K-250K (estimate) Mid-level Full Time
Tasks
- Bridge research infrastructure and production systems
- Build manage and grow ML systems and infrastructure engineering team
- Build scalable serving infrastructure for concurrent workloads
- Collaborate with ML researchers compiler teams and cloud platform teams
- Deliver inference features structured outputs sampling strategies and performance optimization
- Design and scale high throughput low latency inference pipelines
- Drive complex cross functional execution across ML engineering compiler runtime and cloud infrastructure
- Ensure high quality releases via testing validation and operational rigor
- Identify and prioritize technical debt and system bottlenecks
- Improve latency throughput and compute efficiency
- Lead multimodal model execution text image audio video
- Maintain inference reliability and observability across inference stack
- Own ML inference runtime and serving systems architecture
- Partner with cloud compiler runtime hardware and ML teams to optimize performance
- Provide technical direction mentorship and career development
- Recruit talent in ML systems distributed systems and runtime engineering
Perks/Benefits
- N/A
Skills/Tech-stack
C++ | Cloud infrastructure | Deep learning | Distributed Systems | High Performance | High-Performance Computing | Inference Optimization | LLM serving | Latency optimization | Machine Learning | Microservices | Model Execution Pipelines | Model execution | Observability | Performance Computing | Performance Tuning | PyTorch | Python | Reliability Engineering | TensorRT-LLM | Testing | Throughput Optimization | VLLM | Validation
Education
N/A
Regions
Countries
States
Related jobs
-
Classification Algorithms | Data Analysis | Deep learning | Language Models | Language ProcessingSenior-level Full TimeSan Jose, California, United States13h ago
-
Software Engineer Manager II, Embedded Systems, Firmware USD 207K-300KAgile project management | Automated testing | C++ | Direct memory access | Embedded operating systemsSenior-level Full TimeSunnyvale, CA, USA14h ago
-
Attribution Modeling | Causal Inference | Customer Lifetime Value | Customer Segmentation | Difference-in-differences401k match | Annual bonus | Company equipment provided | Company subsidized medical dental and vision | Disability benefitsSenior-level Full TimeAtlanta, GA preferred, Remote R23h ago
-
AI & Analytics Consulting Manager CAD 120K-170KAPI Design | Agile | BPMN | Camunda | Event DrivenCareer developmentSenior-level Full TimeOntario, Canada - Remote R1d ago
-
Advanced Analytics Manager USD 117K-209KAzure | Business Intelligence | Dashboarding | Data Governance | Data ModelingMid-level Full TimeCambridge - B3 Crossing, United States1d ago
-
Associate Director, Computational Biology USD 180K-240KAgent-based | Agent-based modeling | Bioinformatics Databases | Cell biology | D3JS401k | Dental insurance | ESPP | Employee wellness | Medical insuranceMid-level Full TimeSilver Spring, MD, United States1d ago
-
Manager, Data Science USD 60K-75KAttribution | Data Mining | Data Preparation | Data Transformation | Explainable AIHybrid work model | Visa sponsorshipMid-level Full TimeMalvern, PA, United States1d ago
-
Manager, Data Science USD 91K-118KData Products | Elasticsearch | Experimentation | Language Processing | Machine LearningHealth benefits | Wellness benefitsMid-level Full TimeDetroit - One Campus Martius, United …1d ago
-
Senior Manager, Analytics & Insights USD 82K-182KData Modeling | Data Visualization | Data Warehousing | Python | Query OptimizationHybrid work environment | Medical, dental, and vision coverage | Paid time off | Retirement savings options | Wellness programsSenior-level Full TimeHartford-Farmington Ave Atrium, United States1d ago
-
Senior Manager of Software Engineering for Data Platform USD 175K-185KAWS | Access Control | Alert Suppression | Alert escalation | Alert routingSenior-level Full TimeJersey City, NJ, United States1d ago
-
Senior Cybersecurity Analytics Manager USD 116K-184KBig Data | Cloud Computing | Cybersecurity monitoring | Data Transformation | Data analyticsFederal holidays off | Flexible PTO | Professional development support | Tuition reimbursement | Wellness stipendsSenior-level Full TimeWashington, D.C. Metro1d ago
-
Data Science Manager - Sports Pricing USD 160K-190KMachine Learning | Optimization | Python | R | Risk Management401k with company match | Company equipment provided | Company in person events | Company subsidized medical dental vision | Disability benefitsMid-level Full TimeAtlanta, GA preferred, Remote R1d ago
-
Data Science Manager - Market Origination USD 160K-190KData Science | Experimentation | Forecasting | Machine Learning | Predictive Modeling401k match | Annual performance reviews | Career development opportunities | Company equipment | Company in person eventsMid-level Full TimeAtlanta, GA preferred, Remote R1d ago
-
Manager- Applied Sciences / Machine Learning USD 163K-331KApache Spark | Artificial Intelligence | Artificial Intelligence Generated Content | C# | C++Mid-level Full TimeRedmond, WA, US; Mountain View, CA, …1d ago
-
CNBC News Analytics Manager USD 90K-110KAdobe Analytics | Data Visualization | Domo | Microsoft Excel | Microsoft PowerPoint401k | Dental insurance | Employee discounts | Medical insurance | Paid leaveMid-level Full TimeNew York, NEW YORK, United States1d ago
-
Principal Data Engineer USD 152K-190KApache Spark | Artificial Intelligence | CI/CD | Cloud Platforms | Code Coverage401k company match | Dental insurance | Flexible paid time off | Life insurance | Long-term disabilitySenior-level Full TimeDallas, TX - Hybrid (3x in … R1d ago
-
Engineering Manager CAD 138K-200KAI guardrails | Agentic Workflows | Code review | Debugging | Information RetrievalExtended health benefits | RRSP matching | Remote work (Canada) | Stock options | Unlimited paid vacationMid-level Full TimeCanada, Remote R1d ago
-
Senior Manager, Analytics USD 140K-160KAnalytics | Artificial Intelligence | Automation | Customer Data | Data AnalysisDental coverage | Employee discounts | Employee equity | Medical coverage | Pet insuranceSenior-level Full TimeRemote - United States R1d ago
-
3D Scene | 3D Scene Understanding | Autolabel Pipelines | BEV | C++401k match | Dental insurance | Disability insurance | Health insurance | Learning and wellness stipendsSenior-level Full TimeSunnyvale, California, United States1d ago
-
Cross-Functional Collaboration | Cross-functional | Functional collaboration | Generative AI | MLOpsEmployee discounts | Employee equity | Medical, dental & vision coverage | Pet insurance | Stock purchase planSenior-level Full TimeRemote - United States R1d ago
-
Strategy & Execution Manager, GTM USD 130K-223KAgent Orchestration | Apache Spark | Artificial Intelligence | Cloud Data | Cloud data warehousingMid-level Full TimeUnited States1d ago
-
Manager of Data Platform Engineering USD 87K-119KAgile | Backlog Grooming | Capacity Planning | Cloud infrastructure | Data EngineeringSenior-level Full TimeMorristown, NJ, United States1d ago
-
Director, Analytics Engineering USD 270K-330KAggregation | Airflow | BigQuery | DBT | Data Governance401k plan | Commuter benefits | Employee assistance program | Fitness benefits | Flexible time offExecutive-level Full TimeNew York, NY1d ago
-
Director of Engineering, AI & Computer Vision USD 200K-231KAWS | Call Management | Cloud Architecture | Computer Vision | Data EngineeringExecutive-level Full TimeAlpharetta, GA1d ago
-
AI/ML Engineering Manager CAD 152K-234KAWS Bedrock | AWS CDK | AWS CloudFormation | AWS Lambda | AWS SageMakerEquipment and office stipend | Flexible PTO | Fully remote | Learning and development stipend | Medical insuranceMid-level Full TimeCANADA R1d ago