Engineering Manager, Inference Routing and Performance
San Francisco, CA | New York City, NY
USD 405K-485K Mid-level Full Time
Tasks
- Build quantitative performance modeling culture
- Coach engineers through shifting roadmap priorities
- Create technical strategy for routing across heterogeneous accelerators
- Develop and retain distributed systems engineers
- Drive deploy safety incident response on call
- Hire and evaluate senior technical candidates
- Investigate tail latency regressions
- Own technical roadmap for cluster level inference efficiency
- Partner to identify fleet level throughput and latency wins
- Run post incident review and process changes
- Sequence competing infrastructure priorities
Perks/Benefits
- Flexible hybrid office policy
- Flexible working hours
- Generous vacation
- Optional equity donation matching
- Parental leave
- Visa sponsorship
Skills/Tech-stack
Cache Management | Deploy Safety | Distributed State | Distributed Systems | High Performance | High-performance networking | Incident Response | Latency optimization | Load Balancing | On-Call | On-call operations | Performance Modeling | Scheduling | Throughput Optimization
Education
Roles
Regions
Countries
States
Related jobs
-
Engineering Manager, Software Engineering USD 207K-300KCompilers | Data Compression | Developer tools | Distributed Systems | Infrastructure EngineeringSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Senior Software Engineering Manager, Storage, AI/ML USD 262K-365KAI | Benchmarking | Cloud Storage | Compute Technologies | Data ProcessingSenior-level Full TimeSeattle, WA, USA5h ago
-
API Development | AWS | Azure | Cloud platform | Compliance401k plan | Childcare discounts | Commuter benefits | Discounts at Marriott properties | Employee assistance planSenior-level Full TimeBethesda, MD, United States18h ago
-
VP, Engineering USD 185K-210KAWS | Apache Airflow | Azure Cosmos | Azure Cosmos DB | Cloud ComputingHybrid work option | Remote work optionExecutive-level Full TimeUnited States R1d ago
-
Robotics / FIRST LEGO League (FLL) Project Manager USD 84K-127KEvent planning | Logistics | Media management | Program Management | Project ManagementCollaborative environment | Full-time employment | Growth opportunities | On-site workMid-level Full TimeWright Patterson AFB, OH, United States1d ago
-
IT Manager - Anchorage, AK USD 80K-100KActive Directory | Cause analysis | Contract Management | Desktop Support | Enterprise DesktopEmployee referral programMid-level Full TimeAnchorage , AK, United States1d ago
-
AWS | Automation | Call Management | Hardware Diagnostics | Infrastructure OperationsMid-level Full TimeSterling, Virginia, USA1d ago
-
Senior Technical Program Manager, Data Platform USD 200K-322KAirflow | Batch Processing | Cost Allocation | Data Governance | Data PipelinesSenior-level Full TimeUS, CA, Santa Clara, United States1d ago
-
Sr. Engineering Manager - Photon Team (San Francisco) USD 192K-264KDatabase Storage | Database systems | Distributed Systems | Leadership | Query ExecutionSenior-level Full TimeSan Francisco, California1d ago
-
Engineering Manager – Snowtrail USD 236K-339KC++ | Data Structures | Data Structures and Algorithms | Database Query | Database Query EngineOn-site workSenior-level Full TimeUS-WA-Bellevue1d ago
-
Senior Vice President, Applied AI Product Manager USD 104K-253KAPI Design | APIs | APM | Agentic Workflows | Artificial IntelligenceFlexible work resources | Paid leave | Paid volunteer time | Wellbeing programsSenior-level Full TimeNew York, NY, United States2d ago
-
Benchmarking | Code review | Data Pipelines | Distributed Systems | Evaluation metricsMid-level Full TimeMenlo Park, CA2d ago
-
Engineer Manager II, Annotations and Inference Platform USD 207K-300KArtificial Intelligence | Compute Technologies | Data Formats | Distributed Systems | GCSSenior-level Full TimeSunnyvale, CA, USA2d ago
-
Program Manager - Robotics & Computer Vision USD 118K-197KBudget Management | Computer Vision | Dependency management | Integration Management | Milestone planningDental insurance | Disability benefits | Employee assistance program | Health Coach | Life insuranceEntry-level Full TimeNiskayuna, United States2d ago
-
Engineering Manager, Search Storage USD 217K-303KApproximate Nearest Neighbors | Backend Infrastructure | Distributed Systems | Full-text search | KubernetesComprehensive healthcare | Family planning support | Flexible vacation | Gender-affirming care | Mental health coachingMid-level Full TimeRemote - United States R2d ago
-
Senior Product Manager, Replication & Storage Engines USD 118K-231KB-Tree | Cache Management | Cause analysis | Checkpointing | Database replication401k plan | Employee stock purchase program | Equity | Fertility and adoption assistance | Flexible paid time offSenior-level Full TimeNew York City; Seattle2d ago
-
Manager, Machine Learning Operations (MLOps) USD 170K-230KAWS | ArgoCD | CI/CD | Cost Optimization | DBT401k match | FSA options | Flexible PTO | Flexible hybrid work schedule | Life insuranceMid-level Full TimeMarina del Rey, CA2d ago
-
Manager II, Engineering - Bring Your Own Cloud USD 234K-300KAmazon Simple Storage Service | Amazon Web Services | Azure | Cloud infrastructure | Cloud platformCommunity talks | Employee resource groups | Fitness reimbursements | Hybrid workplace | Inclusive company cultureMid-level Full TimeNew York, New York, USA3d ago
-
Data Platforms Manager USD 56K-105KAgile | Automation | Business Intelligence | Data Engineering | Data Lifecycle Management401k plan | Dental insurance | Disability insurance | Life insurance | Medical insuranceSenior-level Full TimeCharlotte NC - 2320 Cascade Pointe …3d ago
-
Senior Data Governance Professional USD 140K-160KBudgeting | Change Management | Compliance | Dashboard Reporting | GovernanceCompany-Paid Holidays | Dental insurance | Disability insurance | Employee assistance program | Life insuranceSenior-level Full TimeNew York, NY, US3d ago
-
Director of AI Engineering USD 198K-250KA/B | A/B Testing | API Versioning | AWS | Agent Orchestration401k match | Annual company retreats | Medical, dental, vision benefits | Paid time off | Promote from withinExecutive-level Full TimeUnited States - Remote R3d ago
-
Senior Product Manager, User Data Platform USD 138K-213KAPIs | B2B SaaS | Data Governance | Data Modeling | Distributed Systems401k | Balance days | Commuter benefits | Dental insurance | Fertility and adoption assistanceSenior-level Full TimeAtlanta, Georgia; Austin, Texas; Boston, Massachusetts; … R5d ago
-
Head of Data, Platform and Intelligence USD 249K-330KAPI Design | Agentic AI | Analytical warehouses | Batch Processing | Cloud Platform (GCPHybrid workExecutive-level Full TimeReston, VA OR Somerville, MA6d ago
-
Project Manager - Robotics USD 90K-130KAutomation | Bill of Materials | Budget Management | Design interpretation | Electrical Design401k match | Dental insurance | Employee assistance program | Health insurance | Life insuranceMid-level Full TimeNoblesville, IN, 46060, US6d ago
-
Machine Learning Engineering Manager - Music USD 184K-262KDeep learning | Distributed Systems | Experimentation systems | JAX | Machine Learning401k retirement plan | Flexible holidays | Health insurance | Meal allowance | Paid parental leaveMid-level Full TimeNew York, NY6d ago