Staff Engineer, Inference Optimizations
Tasks
- Advise on GPU software stacks
- Apply parallelization across multi node GPU clusters
- Collaborate with product and TPMs to ship inference features
- Contribute to open source AI performance projects
- Design benchmarking and performance optimizations
- Develop quantization techniques
- Identify kernel fusion opportunities
- Implement attention layer optimizations
- Improve batch size performance
- Integrate and build open source software
- Lead inference performance architecture
- Lead technical mentorship through code reviews
- Manage memory and precision
- Optimize GPU kernels
- Perform systems design for low level GPU programming
- Tune mixture of experts router kernels
Perks/Benefits
- Conference reimbursement
- Employee assistance program
- Flexible time off
- LinkedIn Learning access
- Local Employee Meetups
- Remote work
- Training reimbursement
Skills/Tech-stack
AMD GPU | Attention Optimization | Bandwidth Optimization | Benchmarking | CUDA | CUDA kernels | FP4 | FP8 | FlashAttention | GPU Kernels | INT8 | Kernel Fusion | MOE | Memory bandwidth | Memory bandwidth optimization | Mixture of Experts | NVIDIA GPU | Parallel Computing | Precision Management | Quantization | ROCm | System design | Tensor cores | TensorRT | Triton
Education
N/A
Related jobs
-
Machine Learning Research Engineer USD 200K-330KAWS | Azure | Benchmarking | CUDA | DDP401k match | Dental insurance | Health insurance | Paid time off | Vision insuranceMid-level Full TimeEmeryville, California, United States; Hybrid (2-3 … R16h ago
-
Staff Engineer, Inference Optimizations USD 191K-239KBF16 | Bandwidth Optimization | CUDA | Expert Routing | FP4Conference reimbursement | Education reimbursement | Employee assistance program | Flexible time off | LinkedIn Learning accessSenior-level Full TimeDenver R16h ago
-
Staff Engineer, Inference Optimizations USD 191K-239KAttention Mechanisms | BF16 | CUDA | CUDA kernels | Distributed SystemsConference reimbursement | Education reimbursement | Employee assistance program | Flexible time off | LinkedIn LearningSenior-level Full TimeBoston R16h ago
-
Staff Engineer, Inference Optimizations USD 191K-239KAttention Optimization | BF16 | CUDA | FP4 | FP8Conference reimbursement | Education reimbursement | Employee assistance program | Employee stock purchase program | Equity compensationSenior-level Full TimeSan Francisco R16h ago
-
Senior Software Engineer I - AI Inference Data Plane USD 139K-174KAutoscaling | Continuous batching | Data parallelism | Distributed Systems | GRPCConference reimbursement | Education reimbursement | Employee assistance program | Employee stock purchase program | Equity compensationSenior-level Full TimeAustin R17h ago
-
Senior Engineer, Inference Data Plane USD 139K-174KContinuous batching | Data parallelism | Distributed Systems | GRPC | GoEmployee assistance program | Employee stock purchase program | Equity compensation | Flexible time off | LinkedIn Learning accessSenior-level Full TimeDenver R17h ago
-
Senior Engineer, Inference Data Plane USD 139K-174KContinuous batching | Data parallelism | Distributed Systems | GRPC | GoConference reimbursement | Employee assistance program | Employee stock purchase program | Equity compensation | Flexible time offSenior-level Full TimeBoston R17h ago
-
Senior Software Engineer I - AI Inference Data Plane USD 139K-174KAutoscaling | Continuous batching | Data parallelism | Distributed Systems | GRPCConference reimbursement | Employee assistance program | Employee stock purchase program | Equity compensation | Flexible time offSenior-level Full TimeSan Francisco R17h ago
-
Senior Embedded Software Engineer - Future Forward USD 134K-201KAuthentication | C# | C++ | CAN | CUDASenior-level Full TimeSunnyvale, CA, United States R1d ago
-
Applied AI Lead USD 120K-150KAgent workflows | Automated evals | Benchmarking | Continuous Monitoring | Evaluation Frameworks401k plan | Health insurance | Paid time off | Parental leave | Visa sponsorship not availableSenior-level Full TimeRemote/Hybrid R2d ago
-
Asynchronous training | Containerization | Curriculum learning | Distributed Systems | Distributed TrainingSenior-level Full TimeSF Bay Area, CA, Remote, US, … R2d ago
-
Principal Machine Learning Engineer, Content Safety USD 295K-345KComputer Vision | Content Moderation | Data Pipelines | Deep learning | Language ModelsEquity compensationSenior-level Full TimeSan Mateo, CA, United States R3d ago
-
Sr Staff Software Engineer- Data Platform USD 220K-255KAWS EMR | AWS Lambda | AWS S3 | Access Control | AlgorithmsDental insurance | Disability insurance | Flexible spending account | Health insurance | Health savings accountSenior-level Full TimeCalifornia - Remote Office, United States R3d ago
-
Principal Embedded Software Engineer USD 150K-175K4G | Battery Management | Battery Management Systems | Board Bring-up | Bring-upSenior-level Full TimeMASSACHUSETTS - VIRTUAL - MA01, United … R3d ago
-
Generative AI Engineer USD 100K-150KAdapters | Benchmarking | DPO | Deep learning | Distributed TrainingSenior-level Full TimeUnited States - Remote R3d ago
-
Senior-level Full TimeUnited States - Remote R3d ago
-
LLM Engineer USD 100K-150KAdapter based Fine Tuning | Attention Optimization | Cluster management | DPO | Distributed TrainingSenior-level Full TimeUnited States - Remote R3d ago
-
AI Optimization Engineer USD 100K-150KBenchmarking | C++ | CUDA | Continuous batching | CutlassRemote workSenior-level Full TimeUnited States - Remote R3d ago
-
Data Engineer, Principal (Hybrid) USD 170K-190KAgile | Azure Synapse | Azure Synapse Analytics | Bitbucket | CI/CDContract-to-hire opportunity | Hybrid work scheduleSenior-level ContractSan Francisco, CA R3d ago
-
AWS | AWS CDK | AWS CodeBuild | AWS CodePipeline | AWS Secrets401k | Healthcare coverage | PTO | Paid Company Holidays | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R3d ago
-
Senior AI Engineer, Contract To Hire USD 130K-195KAWS | Agentic AI | Cloud Native | Cloud platform | Cloud-native developmentSenior-level Full TimeRemote, United States R3d ago
-
Sr. AI Platform Engineer USD 105K-180KAWS | Airflow | Automation | Dagster | DockerDental insurance | Health insurance | Holidays | Paid time off | Professional development opportunitiesSenior-level Full TimeRichardson, TX, United States R4d ago
-
Sr. AI Platform Engineer HYBRID USD 105K-180KAI Platform | AI Workflow Orchestration | AI workflow | AWS | AirflowDental insurance | Health insurance | Paid Holidays | Paid time off | Professional developmentSenior-level Full TimeRichardson, TX, United States R4d ago
-
Edge AI Engineer USD 100K-150KC++ | Core ML | DSP | Embedded Systems | Energy optimizationCareer growth | Remote workSenior-level Full TimeUnited States - Remote R4d ago
-
Senior-level Full TimeUnited States - Remote R4d ago