LLM Inference Frameworks and Optimization Engineer
San Francisco, Singapore, Amsterdam
USD 160K-230K Mid-level Full Time
Tasks
- Analyze inference performance bottlenecks
- Apply CUDA graph optimizations
- Design distributed inference engines
- Develop model execution plans
- Implement distributed inference strategies
- Implement speculative decoding
- Optimize GPU TPU and accelerator performance
- Optimize TensorRT and TRT-LLM graphs
- Optimize end to end model serving pipelines
- Optimize inference latency and throughput
- Perform software hardware co design
- Use torch compile for model execution
Perks/Benefits
Skills/Tech-stack
C++ | CUDA | CUDA graph | Cluster scheduling | Compiler | Efficient kernels | GPU Cluster | GPU Kernel | GPU Programming | GPU cluster scheduling | GPU kernel optimization | KV cache | Kernel optimization | Mixture of Experts | Model Quantization | Pipeline parallelism | PyTorch | Python | Speculative decoding | TRT-LLM | Tensor Parallelism | TensorRT | Torch compile | Transformer | Triton | Workload Scheduling
Education
N/A
Regions
Countries
States
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R21d ago
-
Consultant- Marketing Data Science & AI USD 98K-122KAWS | Azure | Cloud platform | Clustering | Data Engineering401k match | Cell phone stipend | Employee assistance program | FSA | HSAMid-level Full TimeSan Francisco, CA, United States8h ago
-
Senior-level Contract Full TimeLavallette, NJ, United States9h ago
-
Senior AI/ML Data Engineer USD 162K-226KAWS | Apache Airflow | BigQuery | CI/CD | Cloud ComposerCareer growth | Flexible remote days | In office five days per weekSenior-level Full TimeFrisco, TX, United States R11h ago
-
Embedded Software Engineer– Technical Lead USD 150K-175KARM | Bring-up | C++ | CI/CD | Debugging401k match | Health insurance | Paid time offSenior-level Full TimeWaltham, MA, United States12h ago
-
Mid-level ContractBeaverton, OR, US13h ago
-
Manager, AI Engineering USD 49KArtificial Intelligence | CI/CD | Computer Vision | Deep learning | DockerMid-level Full TimeMorrisville, North Carolina, États-Unis d’Amérique15h ago
-
Senior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL16h ago
-
MLOps Engineer ID72409 USD 125K-193KCI/CD | Cloud Computing | Concept drift | Concept drift detection | Data DriftAnnual learning budget | Collaborative culture | Flexible hours | Internal TechTalks | MentorshipMid-level Full TimeArlington, United States R16h ago
-
Azure DevOps | Data Modeling | Data integration | ETL | GitHubRemote workSenior-level ContractSaint Paul, United States16h ago
-
Agile | Data analytics | Distributed Systems | ETL | Machine Learning401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimeNew York16h ago
-
Senior-level Full TimeBoston, Massachusetts, United States16h ago
-
Lead Applied AI Site Reliability Engineer II - PxE A&A USD 113K-232K.NET | ArgoCD | Automation | Bash | C#Mid-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …16h ago
-
.NET | AWS | Agile | Amazon Web Services | AngularSenior-level Full TimeGilbert, Arizona, United States; Lake Mary, …16h ago
-
Mid-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …16h ago
-
Palantir Foundry Data Engineer USD 84K-140KAgile | Batch Processing | Cloud Computing | Continuous integration | Data LineageMentorship | Professional development | Travel 10% averageMid-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …16h ago
-
Lead Applied AI Engineer II USD 124K-255K.NET | AWS | AWS Bedrock | Agent Orchestration | Agentic AISenior-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …16h ago
-
.NET | AI | AI SSDLC | Agentic Workloads | Argo CDMid-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …16h ago
-
Senior-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …16h ago
-
Senior AI Engineer USD 142K-186KAWS | Agent systems | Azure | Containerization | Database Management SystemSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL17h ago
-
Senior Software Engineer, Fleet-level ML Performance USD 174K-252KC++ | Computer Architecture | Deep learning | Embedding architectures | Machine LearningSenior-level Full TimeSunnyvale, CA, USA18h ago
-
API Integration | Agent systems | BigQuery | CI/CD | CrewAIEquity compensation | Health insurance | Paid time off | Retirement planSenior-level Full TimeSunnyvale, CA, USA18h ago
-
Big Data | Data Governance | Data Lifecycle Management | Data Management | Data ModelingSenior-level Full TimeAustin, TX, USA18h ago
-
Forward Deployed Engineer IV, GenAI, Google Cloud USD 207K-300KAPIs | Agent systems | Cloud platform | Cost Per | Cost Per RequestSenior-level Full TimeNew York, NY, USA; Austin, TX, …18h ago
-
Agentic Workflows | C++ | Data Processing | Data Storage | Data StructuresMid-level Full TimeMountain View, CA, USA; New York, …18h ago