Senior Machine Learning Engineer, LLM Inference Optimization
Palo Alto, California, United States
USD 195K-262K Senior-level Full Time
Tasks
- Benchmark inference performance
- Build reproducible benchmark harnesses
- Debug model quality and performance regressions
- Deploy and extend inference engines
- Design and document performance reports and rollout plans
- Implement model compression workflows
- Implement quantization workflows
- Implement speculative decoding and draft models
- Optimize KV cache and attention memory
- Optimize LLM VLM endpoints
Perks/Benefits
- 401k match
- Career growth
- Collaborative culture
- Disability insurance
- Flexibility
- Health insurance
- International environment
- Learning opportunities
- Life insurance
- Ownership
- Parental leave
- Remote Work Reimbursement
Skills/Tech-stack
AWQ | Benchmarking | CUDA | CUDA kernels | Chunked prefill | Continuous batching | Disaggregated Prefill | Disaggregated prefill decode | FP4 | FP8 | GPTQ | INT4) | INT8 | Inference Server | KServe | KV cache | Knowledge Distillation | LLM | Latency optimization | Nvidia Dynamo | Prefill Decode | Prefix caching | PyTorch | Python | Quantization | Quantization aware training | Ray Serve | SGLang | SmoothQuant | Speculative decoding | TensorRT-LLM | Throughput Optimization | Transformer | Triton | Triton Inference | Triton Inference Server | VLLM | VLM
Education
N/A
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R13d ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | ClickHouse | Data Contracts | Data Modeling401k | Catered lunches | Commuter benefits | Continued education benefit | Equity compensationSenior-level Full TimeNew York, New York, United States4h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | Data Contracts | Data Modeling | Data Quality401k | Catered lunches | Commuter benefits | Company wellness day off | Continued education benefitSenior-level Full TimeLos Angeles, California, United States4h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | Data Contracts | Data Modeling | Data Quality401k | Catered lunches | Commuter benefits | Continued education benefit | FSASenior-level Full TimeSan Francisco, California, United States4h ago
-
AI Engineer USD 113K-180KAngular | Apache Kafka | CI/CD | Context Management | DevOps401k matching | Charitable matching | Community outreach groups | Flexible paid time off | Life insuranceMid-level Full TimeLeawood, KS, US R5h ago
-
Applied AI Engineer USD 80K-120KAPIs | Agent Orchestration | Data Pipelines | Evaluation | FedRAMPDental insurance | Equity compensation | Health insurance | Travel benefits | Vision insuranceMid-level Full TimeSF Bay Area, CA, USA5h ago
-
AI Engineer USD 230K-315KAPI Design | Caching | Concurrent programming | Context Window Pruning | Context caching401k matching | Comprehensive medical/dental/vision coverage | Fertility benefits | Flexible PTO | Lunch CreditSenior-level Full TimeSan Francisco, CA11h ago
-
Senior-level Full TimeColumbia, MD12h ago
-
Software Engineer - Platform & Application USD 175K-315KAPI serving | AWS | Agent Frameworks | Apache Airflow | Apache FlinkCollaborative team environment | Mentorship | Production responsibilityMid-level Full TimeSanta Clara HQ12h ago
-
C++ | Computer Vision | Data pipeline | Deep learning | EvaluationSenior-level Full TimeMountain View, California (HQ)12h ago
-
Senior Embedded Software Engineer - Future Forward USD 134K-201KAuthentication | C# | C++ | CAN | CUDASenior-level Full TimeSunnyvale, CA, United States R12h ago
-
Applied AI Lead USD 130K-150KAI Agents | AWS | AWS Bedrock | Anthropic Claude | Azure401k match | Dental insurance | EAP | Life and ADandD insurance | Medical insuranceSenior-level Full TimeHouston, TX, United States12h ago
-
Power Platform Analytics Engineer USD 100K-203KAI Builder | AI Search | Azure AI | Azure AI Search | Azure DataSenior-level Full TimeWashington, DC13h ago
-
Batch ML serving | DPO | Deep learning | Fine Tuning | InferenceSenior-level Full TimeUS-CA-Menlo Park13h ago
-
Senior Data Engineer - Modern Data & AI Platforms USD 156K-204KAWS | DBT | Data Governance | Data Ingestion | Data ModelingRemote workSenior-level Full TimeKansas City, MO, United States13h ago
-
Production AI Engineer USD 145K-165KAWS | AWS CloudFormation | Automation | CI/CD | Cloud Orchestration401k plan | Basic life insurance | Company match | Employee stock purchasing plan | Healthcare benefitsMid-level Full TimeAtlanta, GA, United States14h ago
-
Systems Engineer USD 120K-197KAgile | Amazon ECS | Amazon EKS | Amazon Web Services | Bash100 percent paid healthcare and dental insurance | 401k matching | Escalating vacation time | Paid parental leave | Professional development trainingSenior-level Full TimeGaithersburg, MD14h ago
-
Data Analyst, Financial Data Engineering USD 161K-242KAirflow | Alerting | Apache Spark | Code review | Data Modeling401k plan | Company bonus | Equity | Medical, dental, vision benefits | Wellness stipendsSenior-level Full TimeNew York16h ago
-
Senior AI Engineer (m/f/d) USD 100K-140K.NET | AI Search | AIOps | API Integration | AWS100 percent salary during sick leave | 4 weeks remote work per year | Childcare support | Coursera access | Experience daysSenior-level Full TimeMadrid, MD, Spain R17h ago
-
Intelligent Systems Engineer II USD 92K-151KAPIs | AWS | Artificial Intelligence | Automation | Backend Development401k match | Community service day | Flexible spending accounts | Life insurance | Medical/Dental/VisionMid-level Full TimeWest Chester, OH, United States17h ago
-
Data Engineer USD 105K-140KAPIs | AWS | Apache Airflow | Apache Iceberg | Apache Kafka401k plan with company match | Annual bonus | Company equipment provided | Company events and team outings | Company medical dental and vision plansMid-level Full TimeAtlanta, GA preferred, Remote R19h ago
-
Azure DevOps | Data Modeling | Data integration | ETL | GitHubSenior-level ContractSaint Paul, United States19h ago
-
Senior Machine Learning Engineer USD 150K-200KAPI Development | AWS SageMaker | Data Pipelines | FastAPI | Feature EngineeringSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL19h ago
-
Junior/Middle Computer Vision Engineer ID72410 USD 98K-159KBenchmarking | Cloud Computing | Computer Vision | Data Annotation | Data QualityEducation budget | Fitness budget | Flexible schedule | Mentorship | Remote and office optionsEntry-level Full TimeAddison, United States20h ago
-
Junior/Middle Computer Vision Engineer ID72410 USD 98K-159KCloud Computing | Computer Vision | Image Processing | Machine Learning | Model EvaluationFlexible schedule | Mentorship | Office work options | Personalized growth roadmaps | Remote work optionsEntry-level Full TimeRichardson, United States20h ago