Tech Lead Manager, Inference
Tasks
- Architect and build serving stack
- Build scheduling and queueing systems
- Debug production incidents
- Integrate serving with online reinforcement learning
- Lead grow and develop inference engineering team
- Manage and optimize inference workloads
- Own SLOs and unit economics
- Own hiring coaching and career growth
- Partner with research to ship model architectures
- Set technical roadmap for serving platform
Perks/Benefits
- N/A
Skills/Tech-stack
Autoscaling | Cache Management | Caching | Continuous batching | Deployment Pipelines | Distributed Systems | Infiniband | KV cache | KV-cache management | Kubernetes | NVLink | Observability | Online Reinforcement Learning | Pipeline parallelism | PyTorch | Python | Quantization | RDMA | Ray | Reinforcement Learning | Request Routing | SGLang | Scheduling | Speculative decoding | Tensor Parallelism | TensorRT-LLM | VLLM
Education
N/A
Roles
Related jobs
-
Lead Data Engineer USD 130K-174KAlerting | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage ADLSSenior-level Full TimeColumbus, GA, United States4h ago
-
Senior Manager Analytics USD 201K-201KClaims data | Dashboards | Data Visualization | Data analytics | Data modernization401k | Dental insurance | Disability insurance | Life insurance | Medical insuranceSenior-level Full TimeSouth Jordan, UT, United States16h ago
-
Manager, AI Deployment Engineering — Enterprise (ProServe, Media, Telco, Private Equity) USD 251K-335KArchitecture | Artificial Intelligence | Cloud infrastructure | Cost Optimization | Data GovernanceMid-level Full TimeSan Francisco22h ago
-
Lead Machine Learning Scientist USD 166K-205KData cleaning | Deep learning | Embedding Models | Feature Engineering | Fine TuningCompany-matching 401k | Flexible spending accounts | Medical, dental, vision benefits | Paid parental leave | Paid sick daysSenior-level Full TimeNew York, NY23h ago
-
Manager of Data Engineering USD 140K-160KAWS | Azure | Data Modeling | Data Warehousing | Data pipelineMid-level Full TimeBaltimore, MD23h ago
-
Sr. Engineering Manager, AI Runtime USD 228K-297KCheckpointing | DeepSpeed | Distributed Systems | Elastic Training | FSDPSenior-level Full TimeMountain View, California; San Francisco, California23h ago
-
Lead Data Scientist (5421) USD 103K-171KArtificial Intelligence | Business Intelligence | Data Engineering | Data Modeling | Data QualityHealth insurance | Paid leave | RetirementSenior-level Full TimeFort Washington, MD23h ago
-
Senior Manager, Data Science & Operation Analytics USD 170K-185KA/B | A/B Testing | B testing | Causal Inference | Dashboard Development401k match | Dental insurance | Disability insurance | Employee assistance program | Health insuranceSenior-level Full TimeNorth Bethesda, MD1d ago
-
Manager, Data Analytics USD 158K-224KA/B | A/B Testing | Amazon Redshift | Amazon S3 | AmplitudeEquity | Healthcare coverage | LifeTime Membership | Parental leave | Retirement savings matchMid-level Full TimeSanta Monica, California, United States R1d ago
-
Devops Team Lead USD 160K-190KAlerting | Cloud platform | Dashboards | Google Cloud | Google Cloud Platform401k retirement plan | Birthday off | Cell phone reimbursement | Child care reimbursement | Company-Paid HolidaysSenior-level Full TimeBoston, MA1d ago
-
Executive-level Full TimeCanada1d ago
-
Autonomy | C++ | CPU GPU | CPU/GPU Optimization | DDS401k | Healthcare | Hybrid work | Paid Holidays | Paid time offSenior-level Full TimeSan Carlos - Hybrid R1d ago
-
AWS | AWS Fault Injector | AWS Fault Injector Service | AWS Resiliency | AWS Resiliency HubHybrid work schedule | Relocation assistanceSenior-level Full TimeCharlotte, NC, United States1d ago
-
Senior Manager, Enterprise Data Platform USD 196K-245KAIOps | Automated testing | CI/CD | DBT | Data GovernanceSenior-level Full TimeRemote - USA R1d ago
-
AWS Lambda | Amazon ECS | Amazon EKS | Amazon EMR | Amazon Web ServicesSenior-level Full TimeJersey City, NJ, United States1d ago
-
Embedded Software Engineer– Technical Lead USD 150K-175KARM | ARMv7 | ARMv8 | Agile | Bring-up401k match | Health insurance | Minimal travel | Paid time offSenior-level Full TimeWaltham, MA, United States1d ago
-
Senior-level Full TimeChicago, Illinois, USA R1d ago
-
API Integration | Agile | Artificial Intelligence | Backlog Grooming | Corporate Treasury ManagementExecutive-level Full TimeJersey City, NJ, United States1d ago
-
Lead Data Architect (P4) USD 133K-185KApache Spark | Azure Data | Azure Data Lake | Data Governance | Data Lake401k | Collaborative work environment | Health insurance | Long-term growth | Training and developmentSenior-level Full TimeAustin, TX, 78735, USA1d ago
-
Lead Data Architect (P4) USD 133K-185KApache Spark | Azure | Azure Data | Azure Data Lake | Data Catalog401k | Health insurance | Long-term growth | Supportive collaborative environment | Training and developmentSenior-level Full TimeAustin, TX, 78735, USA1d ago
-
API Design | Agile | Angular | Artificial Intelligence | Automated testingSenior-level Full TimeChicago, IL, United States1d ago
-
1000000545.Data Architect Lead.Info Tech Services USD 96K-120KAPI Integration | AWS | Azure | CI/CD | DAMASenior-level Full TimeTX, United States1d ago
-
Data Science Manager - TikTok Ads USD 153K-296KData Analysis | Data Modeling | Data Pipelines | Experimentation | Machine LearningMid-level Full TimeSeattle, Washington, United States1d ago
-
Data Science Manager - TikTok Ads USD 153K-296KData Analysis | Data Modeling | Data pipeline | ETL | ExperimentationMid-level Full TimeSan Jose, California, United States1d ago
-
AWS | Apache Flink | Apache Spark | Architecture Decision | Architecture Decision RecordsAccess to modern AI development tools | Equity and long term incentive programs | Flexible work environment | Fully remote within Canada | Health dental and wellness benefitsMid-level Full TimeCanada R1d ago