Engineering Manager, Inference Routing and Performance
San Francisco, CA | New York City, NY
USD 405K-485K Mid-level Full Time
Tasks
- Build quantitative performance modeling culture
- Coach engineers through shifting roadmap priorities
- Create technical strategy for routing across heterogeneous accelerators
- Develop and retain distributed systems engineers
- Drive deploy safety incident response on call
- Hire and evaluate senior technical candidates
- Investigate tail latency regressions
- Own technical roadmap for cluster level inference efficiency
- Partner to identify fleet level throughput and latency wins
- Run post incident review and process changes
- Sequence competing infrastructure priorities
Perks/Benefits
- Flexible hybrid office policy
- Flexible working hours
- Generous vacation
- Optional equity donation matching
- Parental leave
- Visa sponsorship
Skills/Tech-stack
Cache Management | Deploy Safety | Distributed State | Distributed Systems | High Performance | High-performance networking | Incident Response | Latency optimization | Load Balancing | On-Call | On-call operations | Performance Modeling | Scheduling | Throughput Optimization
Education
Roles
Regions
Countries
States
Related jobs
-
Capacity Planning | Cloud Computing | Cloud platform | Cross-functional | Cross-functional leadershipSenior-level Full TimeKirkland, WA, USA; Sunnyvale, CA, USA5h ago
-
Manager I, Engineering - APM Retrieval USD 187K-240KAPI Development | Cross-functional | Cross-functional leadership | Distributed Systems | Functional leadership401k match | Dental | Discounted employee stock purchase plan | Fitness reimbursement | HealthcareMid-level Full TimeNew York, New York, USA1d ago
-
Software Engineering Manager, AI/ML, User Identity USD 207K-300KAI | Architectural Design | Cloud Computing | Distributed Systems | IdentityFlexible working hours | Health insurance | Paid time off | Professional development opportunitiesSenior-level Full TimeSunnyvale, CA, USA1d ago
-
C Programming | C++ Programming | Distributed Systems | Infrastructure Design | NetworkingHealth benefits | Insurance | Professional development | Retirement plansSenior-level Full TimeNew York, NY, USA; Sunnyvale, CA, …1d ago
-
Software Engineering Manager, Persistent Disk USD 207K-300KCloud technologies | Distributed Systems | Project Management | Software development | Team leadershipBenefits | Bonus | EquitySenior-level Full TimeKirkland, WA, USA1d ago
-
Group Manager-Data Ops and Data Governance USD 178K-260KAccess Controls | Cause analysis | Cost Management | Data Classification | Data Governance401 K Savings Plan | Coffee service | Education assistance | Fitness Center | Food serviceMid-level Full TimeMT-MF-Red Tag, United States1d ago
-
Engineering Manager - Pipelines Engine USD 190K-261KData Infrastructure | Distributed Systems | Performance optimization | Product Management | Software EngineeringBenefits | Compensation rangeMid-level Full TimeMountain View, California; San Francisco, California1d ago
-
Director of Data Engineering USD 145K-250KAWS | Agile | Bash Scripting | Data Processing | Data Processing Frameworks401k | Disability insurance | Employee assistance program | Flexible paid time off | Flexible spending accountsExecutive-level Full TimeCharlotte, NC R1d ago
-
Director II, Engineering USD 225K-274KCloud Native | Containerization | Distributed Systems | Java | KubernetesExecutive-level Full Time*Job Posting Only: USA12d ago
-
Associate Director, Bioinformatics Operations USD 153K-259KAutomation | Bioinformatics | CAP | CLIA | Capacity PlanningDental benefits | Medical benefits | Paid parental leave | Paid time off | Retirement planMid-level Full Time1 Exact Lane, United States3d ago
-
Engineering Manager, Online Data Systems USD 325K-405KCloud Computing | Data Infrastructure | Database | Distributed Systems | IndexingMid-level Full TimeSan Francisco5d ago
-
Distributed Systems | Go | Kubernetes | Leadership | Software developmentBenefits | Bonus | EquitySenior-level Full TimeRaleigh, NC, USA; Durham, NC, USA6d ago
-
Senior Product Manager - Xbox Data Analytics USD 119K-258KAmbiguity management | Azure Cosmos | Azure Cosmos DB | Azure Data | Azure Data ExplorerSenior-level Full TimeRedmond, WA, US6d ago
-
Senior Engineering Manager, Data Engineering USD 212K-250KAbstraction layers | Access standardization | Backend architecture | Change Data Capture | Communication skillsEducational reimbursements | Stock plan | Unlimited vacation | Wellness reimbursements | Zero-cost insuranceSenior-level Full TimeNew York, NY; San Ramon, CA6d ago
-
Director, Engineering – Applied AI USD 322K-387KAI AI pipelines | AI Pipelines | AI/AI | API Development | Agent OrchestrationExecutive-level Full TimeRemote, United States R6d ago
-
Director, Engineering USD 150K-200KAI | AWS | Anomaly Detection | Azure | BashEquity participation | Health and wellness benefits | Hybrid or remote work | Time off policiesExecutive-level Full TimeUnited States - Remote R7d ago
-
Budget Management | Data Engineering | Data Science | Earned Value | Earned Value ManagementEmployee resource groups | Flexible work arrangements | Health insurance | Paid Holidays | Paid time offSenior-level Full TimeCASDRB08, United States7d ago
-
Data Science Manager USD 148K-308KAI | Data Analysis | Data Compression | Data Engineering | Data MiningCareer development | Healthcare plans | Paid Holidays | Paid time off | Wellbeing programsSenior-level Full TimeBoise, ID - Main Site, United …7d ago
-
Data Engineer, Principal USD 127K-198KAutomation | Cloud Platforms | Data Architectures | Data Governance | Data ModelingSenior-level Full TimeUnited States7d ago
-
Staff Tech Lead Manager, ML Data Services USD 225K-275KBeam | Cloud Platforms | Data Engineering | Data Governance | Data Pipelines401k | Dental insurance | Health saving accounts | Life insurance | Medical insuranceSenior-level Full TimePittsburgh, Pennsylvania, United States7d ago
-
Staff Tech Lead Manager, ML Data Services USD 225K-275KAWS | Azure | Beam | Cloud Platforms | Data Engineering401k | Dental insurance | Health saving accounts | Life insurance | Medical insuranceSenior-level Full TimeBoston, Massachusetts, United States7d ago
-
Staff Tech Lead Manager, ML Data Services USD 225K-275KAWS | Azure | Beam | Data Governance | Data Infrastructure401k | Dental | Health savings accounts | Life insurance | MedicalSenior-level Full TimeBoston, Massachusetts, United States; Pittsburgh, Pennsylvania, … R7d ago
-
Data Engineering Manager, Analytics USD 248K-279KAI | Agile methodologies | Data Architecture | Distributed Systems | ETL PipelinesRemote workMid-level Full TimeMenlo Park, CA | Remote, US R8d ago
-
Software Engineering Manager USD 80K-149KCI/CD | Distributed Systems | High Availability | High-availability architectures | Incident ManagementMid-level Full TimeRemote, United States R8d ago
-
Software Development Manager, Specialist Technology Team USD 184K-250KAPI Development | AWS | Agile methodology | Distributed Systems | Language ProcessingDiversity and inclusion initiatives | Mentorship opportunities | Work-life balance programsMid-level Full TimeAustin, Texas, USA8d ago