Staff Engineer, Inference Optimizations
Tasks
- Advise on GPU software integration
- Benchmark and optimize inference engine
- Contribute to open source AI performance
- Identify kernel fusion opportunities
- Implement attention layer optimizations
- Implement quantization techniques
- Lead inference performance architecture
- Lead technical design reviews and mentorship
- Manage memory and precision for inference
- Optimize GPU kernels for throughput and latency
- Parallelize workloads across multi node GPU clusters
- Tune MoE expert gateway router kernels
- Tune batch size performance
Perks/Benefits
- Conference reimbursement
- Education reimbursement
- Employee assistance program
- Flexible time off
- LinkedIn Learning access
- Local Employee Meetups
- Remote work
- Training reimbursement
Skills/Tech-stack
BF16 | Bandwidth Optimization | CUDA | Expert Routing | FP4 | FP8 | FlashAttention | GPU Architecture | INT8 | Kernel Fusion | Memory bandwidth | Memory bandwidth optimization | Mixture of Experts | OpenAI Triton | Parallel Computing | Quantization | RMS Normalization | ROCm | Tensor cores | TensorRT | Triton
Education
N/A
Related jobs
-
Senior Research Engineer, LLM Training & Post-Training USD 165K-310KCUDA | DPO | DeepSpeed | Distributed Training | Experiment design401k matching | Company holidays | Comprehensive health coverage | Dental coverage | Flexible time offSenior-level Full TimeNew York, New York, United States; … R22h ago
-
AI/ML Engineer, Lead USD 128K-292KAWS | Agent Orchestration | Agentic AI | Autonomous Agents | AzureDependent care | Paid leave | Professional development | Tuition assistance | Work-life programsSenior-level Full TimeUSA, VA, Ashburn (22001 Loudoun County … R1d ago
-
Edge AI Engineer USD 100K-155KArchitecture Optimization | Bias Evaluation | C++ | Core ML | DSPRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
Machine Learning Infrastructure Engineer USD 105K-143KAutoscaling | C++ | Capacity Planning | Distributed Systems | FinOpsSenior-level Full TimeUnited States - Remote R1d ago
-
MLOps Engineer USD 100K-150KAPI Gateway | Abuse detection | Access Management | Automated rollback | AutoscalingSenior-level Full TimeUnited States - Remote R1d ago
-
AI Optimization Engineer USD 100K-100KC++ | Compiler optimization | Cutlass | Deep learning | DeepSpeedRemote workSenior-level Full TimeUnited States - Remote R1d ago
-
ML Performance Engineer USD 100K-150KBenchmarking | C++ | Compiler optimization | Continuous batching | Deep learningRemote workSenior-level Full TimeBellevue, WA R1d ago
-
AWS | AWS Batch | AWS Bedrock | AWS CDK | AWS IoTHealthcare | PTO | Paid Holidays | Phone stipend | Wellness stipendSenior-level Full TimeSan Carlos - Hybrid R2d ago
-
Machine Leaning Performance Engineer (Inference) USD 200K-300KC++ | CUBLAS | CUDA | CUDA C | CutlassFitness Events | Free meals | Hybrid work options | Paid time off | Volunteer opportunitiesMid-level Full TimeNew York R2d ago
-
Edge Computing AI Engineer USD 100K-105KC++ | Edge inference | Embedded Systems | Federated Learning | Machine LearningCareer growthMid-level Full TimeUnited States - Remote R2d ago
-
Edge AI Engineer USD 70K-100KC++ | Edge inference | Embedded Systems | Hardware-Aware Optimization | Machine LearningCareer growth | Remote workSenior-level Full TimeUnited States - Remote R2d ago
-
Data Platform Engineer USD 100K-150KAutoscaling | C++ | Capacity Planning | Distributed Systems | GPU ArchitectureCareer growth potential | Remote workSenior-level Full TimeUnited States - Remote R2d ago
-
Model Serving Engineer USD 74K-98KAutoscaling | C++ | Caching | Capacity Planning | Distributed SystemsCareer growth | Remote workSenior-level Full TimeUnited States - Remote R2d ago
-
Algorithm Engineer (Image Processing) USD 105K-180KC plus plus | CUDA | Computer Vision | Data Analysis | Data Visualization401k matching | Dental insurance | Development and career growth | Employee assistance program | Employee stock purchase programSenior-level Full TimeUSA-MI-Ann Arbor-KLA, United States R2d ago
-
Machine Learning Systems Engineer USD 144K-192KCUDA | GPU Profiling | Kernel Fusion | PyTorch | PyTorch distributed401k match | Dental insurance | Health insurance | Health savings account | Life insuranceSenior-level Full TimeBoston, Massachusetts, United States; Las Vegas, … R3d ago
-
Edge ML Engineer USD 100K-150KC++ | Edge AI | Embedded Systems | Federated Learning | Machine LearningSenior-level Full TimeUnited States - Remote R3d ago
-
AI Platform Engineer USD 130K-180KAPI Gateway | AWS | Alerting | Autoscaling | Azure BicepCareer growth | Remote workSenior-level Full TimeUnited States - Remote R3d ago
-
Staff ML Engineer, Perception Research USD 251K-310K3D Perception | Autoregressive models | Data Pipelines | Data parallelism | Diffusion Models401k match | Baby bonding leave | Dental insurance | Disability insurance | Health insuranceSenior-level Full TimeMountain View, CA, USA ; New … R5d ago
-
Software Engineer, Inference Runtime USD 150K-350KC++ | CPU Profiling | CUDA | Executorch | GPU ProfilingDental insurance | Flexible PTO | Flexible WFH | Medical insurance | Team mealsMid-level Full TimeNew York City R5d ago
-
AI Inference Tools Engineer USD 167K-242KAgent Frameworks | Benchmarking | C++ | CUDA | Dashboards401k matching | Flexible paid time off | Health insurance | Hybrid work options | Local meetupsSenior-level Full TimeUnited States, United Kingdom R6d ago
-
Senior-level Full TimeUnited States - Remote R6d ago
-
ML Systems Engineer USD 145K-165KAutoscaling | C++ | Capacity Planning | Distributed Systems | GPU ArchitectureRemote workSenior-level Full TimeUnited States - Remote R6d ago
-
Mid-level Full TimeUnited States - Remote R6d ago
-
Audio AI Engineer USD 80K-160KASR | Character Error Rate | Data pipeline | Distillation | Error rate401k matching | Career development training via technical and management training | Commuter benefits | Flexible spending account | Flexible time offMid-level Full TimeReston, VA R7d ago
-
Senior-level Full TimeLexington, MA R7d ago