Engineering Manager, Inference ML Runtime
Sunnyvale CA or Toronto Canada
USD 180K-250K (estimate) Mid-level Full Time
Tasks
- Bridge research infrastructure and production systems
- Build manage and grow ML systems and infrastructure engineering team
- Build scalable serving infrastructure for concurrent workloads
- Collaborate with ML researchers compiler teams and cloud platform teams
- Deliver inference features structured outputs sampling strategies and performance optimization
- Design and scale high throughput low latency inference pipelines
- Drive complex cross functional execution across ML engineering compiler runtime and cloud infrastructure
- Ensure high quality releases via testing validation and operational rigor
- Identify and prioritize technical debt and system bottlenecks
- Improve latency throughput and compute efficiency
- Lead multimodal model execution text image audio video
- Maintain inference reliability and observability across inference stack
- Own ML inference runtime and serving systems architecture
- Partner with cloud compiler runtime hardware and ML teams to optimize performance
- Provide technical direction mentorship and career development
- Recruit talent in ML systems distributed systems and runtime engineering
Perks/Benefits
- N/A
Skills/Tech-stack
C++ | Cloud infrastructure | Deep learning | Distributed Systems | High Performance | High-Performance Computing | Inference Optimization | LLM serving | Latency optimization | Machine Learning | Microservices | Model Execution Pipelines | Model execution | Observability | Performance Computing | Performance Tuning | PyTorch | Python | Reliability Engineering | TensorRT-LLM | Testing | Throughput Optimization | VLLM | Validation
Education
N/A
Regions
Countries
States
Related jobs
-
AI machine learning | Amazon Redshift | Amazon Web Services | Cloud Computing | Data GovernanceHealth insurance | Paid time off | Retirement contributionsSenior-level Full TimeBoston, Massachusetts, US, 022104h ago
-
Director, Data Engineering USD 234K-253KAWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon S3Career growth | Collaborative culture | Employee mentoring | Hybrid work | Inclusive work environmentExecutive-level Full TimeSan Diego, California, United States5h ago
-
Senior Product Manager, Analytics Platform USD 136K-204KA/B | A/B Testing | AI | Analytics | Anomaly DetectionSenior-level Full TimeBoston, MA6h ago
-
AI Engineering Lead USD 246K-329KAgentic Workflows | Context Management | Language Models | Language Processing | Large Language ModelsSenior-level Full TimeSF or NYC10h ago
-
Software Engineering Manager, LLM Training USD 170K-277KCUDA | Containerization | Data parallelism | Distributed Systems | DockerFlexible-hybrid work | Health and wellness programs | Time offEntry-level Full TimeMountain View, CA, United States11h ago
-
Director, Data Engineering - League Studios USD 251K-351KAlgorithms | Atlan | DBT | Data Engineering | Data Governance401k match | Dental insurance | Flexible work schedules | Life insurance | Medical insuranceExecutive-level Full TimeLos Angeles, USA12h ago
-
Manager, Data Analytics USD 132K-212KCloud Computing | Data Governance | Data Modeling | Data Reconciliation | Data ValidationFlexible remote days | In-person collaborationSenior-level Full TimePlano, TX, United States R12h ago
-
Senior AI Engagement Manager, Maritime Industrial Base USD 122K-189KAI strategy | Artificial Intelligence | Business Problem Solving | Consulting | Cross-functionalSenior-level Full TimeTysons, Virginia, United States14h ago
-
Senior Machine Learning Engineering Manager USD 345K-399KCloud Computing | Computer Vision | Content Moderation | Data pipeline | Deep learningEquity compensation | Health benefits | Onsite collaboration daysSenior-level Full TimeSan Mateo, CA, United States14h ago
-
Anomaly Detection | CI/CD | Data Engineering | Data analytics | MLOps401k match | Dental insurance | Flexible work schedules | Life insurance | Medical insuranceSenior-level Full TimeLos Angeles, USA15h ago
-
Director of Software Development CAD 177K-220KAPI Integration | Backend Engineering | Cause analysis | Conditional logic | Data ProcessingCo-working space | Health and wellness benefits | Phone and internet subsidy | Professional learning allowance | Remote workExecutive-level Full TimeToronto, ON Hub15h ago
-
Senior Product Manager – Inference CAD 150K-180KAI | API | Data Science | Knowledge graphs | LLMAnnual learning and development budget | Company closures | Flex Time | Health and dental benefits | Home office setup budgetSenior-level Full TimeToronto, Ontario R16h ago
-
Principal Data Engineer - League Studios USD 209K-293KAnalytics Platforms | Apache Airflow | Apache Spark | Batch Data Processing | Batch data401k company match | Dental insurance | Flexible work schedules | Life insurance | Medical insuranceSenior-level Full TimeLos Angeles, USA16h ago
-
Release Manager, Engineering USD 99K-190KAI infrastructure | Agile | Branching strategy | CI/CD | Cause analysisSenior-level Full TimeRedwood City, California, United States16h ago
-
Anti-Money Laundering | Customer risk scoring | Data-driven | Data-driven analytics | Machine LearningSenior-level Full TimeSan Francisco, CA, New York, NY, … R17h ago
-
Engineering Manager, Agentic Systems USD 162K-284KC++ | Deep learning | DeepSpeed | Distributed Training | GPU OptimizationMid-level Full TimeMountain View, CALIFORNIA, United States17h ago
-
Senior Director of Engineering, Traffic and Networking USD 340K-488KAWS | Cloud Computing | Cloud platform | Distributed Systems | Google CloudSenior-level Full TimeUS-WA-Bellevue19h ago
-
Senior Product Manager, Data Platform USD 180K-240K21 CFR | 21 CFR Part 11 | AI Agents | API Design | B2B SaaS401k match | Commuter benefits | Disability coverage | Healthcare dental and vision plans | Life insuranceSenior-level Full TimeBoston, MA19h ago
-
Group Product Manager, Data Platform & Context USD 192K-307KAI Evaluation | APIs | Artificial Intelligence | Data Modeling | Data PipelinesIn person onboarding events | Remote work optionsMid-level Full TimeRemote - USA R19h ago
-
Head of Frontier Data - STEM USD 350K-410KAI Feedback | Continuous integration | Data Quality | Data Synthesis | Data labelingCollaborative environment | Five-day workweek | Flexible working hours | Startup speed execution | Supportive work cultureExecutive-level Full TimeSan Francisco, California, United States; United …20h ago
-
Manager, Marketing Analytics - Migraine USD 137K-223KBusiness Optimization | Data Modeling | Data analytics | Forecasting | Machine Learning401k | Dental insurance | Medical insurance | Paid time off | Short-term incentive planMid-level Full TimeMettawa, IL, United States21h ago
-
Manager, Marketing Analytics - Migraine USD 137K-223KAnalytics | Business planning | Communication | Compliance | Data Analysis401k | Dental insurance | Medical insurance | Paid time off | Short Term Incentive ProgramMid-level Full TimeFlorham Park, NJ, United States21h ago
-
Artificial Intelligence | Big Data | Clinical trial | Clinical trial feasibility | Clinical trials401k | Medical/Dental/Vision insurance | Paid time off | Short Term Incentive ProgramExecutive-level Full TimeNorth Chicago, IL, United States22h ago
-
Director of Data Engineering USD 200K-250KAWS | AWS S3 | Access Control | Amazon Athena | CI/CDAnnual offsite | Equity | Health insurance | Hybrid flexibility | Meal vouchersExecutive-level Full TimeUnited States-Remote R22h ago
-
Data Science Manager USD 160K-205KA/B | A/B Testing | Analytics | B testing | Causal InferenceRemote workMid-level Full TimeEast Coast, US22h ago