Senior Machine Learning Engineer, LLM Inference Optimization
Palo Alto, California, United States
USD 195K-262K Senior-level Full Time
Tasks
- Benchmark inference performance
- Build reproducible benchmark harnesses
- Debug model quality and performance regressions
- Deploy and extend inference engines
- Design and document performance reports and rollout plans
- Implement model compression workflows
- Implement quantization workflows
- Implement speculative decoding and draft models
- Optimize KV cache and attention memory
- Optimize LLM VLM endpoints
Perks/Benefits
- 401k match
- Career growth
- Collaborative culture
- Disability insurance
- Flexibility
- Health insurance
- International environment
- Learning opportunities
- Life insurance
- Ownership
- Parental leave
- Remote Work Reimbursement
Skills/Tech-stack
AWQ | Benchmarking | CUDA | CUDA kernels | Chunked prefill | Continuous batching | Disaggregated Prefill | Disaggregated prefill decode | FP4 | FP8 | GPTQ | INT4) | INT8 | Inference Server | KServe | KV cache | Knowledge Distillation | LLM | Latency optimization | Nvidia Dynamo | Prefill Decode | Prefix caching | PyTorch | Python | Quantization | Quantization aware training | Ray Serve | SGLang | SmoothQuant | Speculative decoding | TensorRT-LLM | Throughput Optimization | Transformer | Triton | Triton Inference | Triton Inference Server | VLLM | VLM
Education
N/A
Regions
Countries
States
Cities
Related jobs
-
Senior Software Engineer, Google Distributed Cloud AI USD 174K-252KBilling | Cloud Networking | Container Orchestration | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Software Engineer III, AI/ML, Foundational Lanes USD 147K-210KAutomation | C++ | Computational Geometry | Computer Vision | Data ProcessingSenior-level Full TimeMountain View, CA, USA3h ago
-
Agent systems | Agentic Frameworks | Backend Development | C plus plus | Data PipelinesSenior-level Full TimeSan Bruno, CA, USA; Mountain View, …3h ago
-
Senior Software Engineer, AI/ML, Google Workspace USD 174K-252KC plus plus | Code Reviews | Data Processing | Data Storage | DebuggingSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Agent systems | C++ | Database technologies | Distributed Systems | Embedding GenerationSenior-level Full TimeChicago, IL, USA; San Bruno, CA, …3h ago
-
Staff Software Engineer, Data Pipelines, DeepMind USD 207K-300KBenchmarking | C++ | Compilers | Data Structures | Data Structures and AlgorithmsBonus | Equity | Health insurance | Learning opportunities | Paid time offSenior-level Full TimeMountain View, CA, USA; New York, …3h ago
-
Software Engineer, On-Device Machine Learning USD 174K-252KAndroid | Compiler development | Core ML | Data Processing | DebuggingBonus | Equity | Health insurance | Paid time off | Retirement planMid-level Full TimeSunnyvale, CA, USA; New York, NY, …3h ago
-
Automation Scripting | Bare Metal | C plus plus | C# | CANHealth insurance | Professional development opportunities | Retirement plansSenior-level Full TimeHuntington Beach, CA13h ago
-
IT Storage Engineer USD 135K-175KAnsible | DFS | Enterprise networking | FlashArray | FlashBlade401k retirement plan | Company shuttles | Employee discounts | Employee stock purchase plan | Life insuranceMid-level Full TimeRedmond, WA15h ago
-
IT Storage Engineer USD 135K-175KAnsible | CIFS | DFS | FlashArray | FlashBlade401k retirement plan | Life insurance | Long-term disability | Medical, dental & vision coverage | Paid HolidaysMid-level Full TimeHawthorne, CA15h ago
-
IT Storage Engineer USD 130K-175KAlerting | Ansible | CIFS | Capacity Planning | Configuration ManagementExtended hours and weekends | On-call rotationMid-level Full TimeStarbase, TX15h ago
-
IT Storage Engineer USD 130K-175KAnsible | CIFS | DFS | FlashArray | FlashBladeExtended hours | On-call rotation | Physical hardware installation | Weekend workMid-level Full TimeCape Canaveral, FL15h ago
-
IT Storage Engineer USD 130K-175KAlerting | Ansible | CIFS | Capacity Planning | Configuration ManagementExtended hours and weekend availability | Occasional physical hardware installation | On-call rotationMid-level Full TimeBastrop, TX15h ago
-
Full Stack Software Engineer, AI & Data Platforms, IS&T USD 143K-250KAWS | Azure | CSS | GCP | GRPCMid-level Full TimeAustin15h ago
-
Software Engineer, Data Integration USD 165K-225KAWS | Automated testing | Azure | Cloud Computing | Data IngestionSenior-level Full TimeRemote - US Only R15h ago
-
Software Engineer - AI Developer Productivity USD 165K-330KAudit Logging | Authentication | Authorization | CI/CD | Context ManagementCompany facilitated 401k | Fertility and family building stipend | Flexible PTO | Medical dental vision insurance 100 percent | Paid parental leaveSenior-level Full TimeSan Francisco R16h ago
-
Telemetry Data Engineer USD 110K-128KAnomaly Detection | Azure Data | Azure Data Explorer | Data Engineering | Data Explorer401k retirement plan | Medical/Dental/VisionMid-level ContractRedmond, Washington, USA R16h ago
-
Software Engineer, Applied AI Infrastructure USD 193K-352KAttention | Batching | CI Failure Attribution | Cloud infrastructure | Context handlingSenior-level Full TimeMountain View, California (HQ)16h ago
-
Senior AI Engineer USD 176K-258KAgentic Workflows | Agile | Cloud Architecture | Code review | DatabricksCell phone allowance | Comprehensive health coverage | Equity grants | Growth-focused environment | Home office setup allowanceSenior-level Full TimeRemote - US R16h ago
-
Senior-level Full TimeAustin, Texas16h ago
-
Machine Learning Engineer - Health AIML USD 139K-304KAgent systems | Deep learning | Generative AI | Huggingface | JAXSenior-level Full TimeSeattle17h ago
-
Sr. Computer Vision Engineer (Deep Learning) USD 180K-240KComputer Vision | Data Augmentation | Deep learning | Distributed Training | Embedded Systems401k | Cell phone stipend | Complimentary meals | FSA | Fertility benefitsSenior-level Full TimeMountain View, CA17h ago
-
Automated testing | Build Automation | C++ | CI/CD | Firmware401k | Employee assistance program | Health savings account | Life insurance | Long-term disabilityMid-level ContractDallas, Texas, USA R17h ago
-
Applied AI Engineer, Office of the CEO USD 140K-170KAgentic Orchestration | Business Intelligence | Data Analysis | Data Governance | Data ModelingMid-level Full TimeNew York, NY17h ago
-
Analytics Engineer II USD 100K-130KBusiness Intelligence | Data Modeling | Data Quality | Data Transformation | Data Warehousing401k program | Dental insurance | Disability insurance | Flexible spending account | Life insuranceMid-level Full TimeChicago, IL17h ago