Senior On-Device Model Inference Optimization Engineer

NVIDIA has been transforming computer graphics, PC gaming, and accelerated computing for more than 25 years. It’s a unique legacy of innovation that’s fueled by great technology—and amazing people.

Today, we’re tapping into the unlimited potential of AI to define the next era of computing. An era in which our GPU acts as the brains of computers, robots, and self-driving cars that can understand the world. Doing what’s never been done before takes vision, innovation, and the world’s best talent. As an NVIDIAN, you’ll be immersed in a diverse, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.

We are seeking a highly-skilled Senior On-Device Model Inference Optimization Engineer to join our team and lead efforts in improving the performance and efficiency of AI models enabling the next generation of autonomous vehicles technology at Nvidia!

What you'll be doing:

• Develop and implement strategies to optimize AI model inference for on-device deployment.

• Employ techniques like pruning, quantization, and knowledge distillation to minimize model size and computational demands.

• Optimize performance-critical components using CUDA and C++.

• Collaborate with multi-functional teams to align optimization efforts with hardware capabilities and deployment needs.

• Benchmark inference performance, identify bottlenecks, and implement solutions.

• Research and apply innovative methods for inference optimization.

• Adapt models for diverse hardware platforms and operating systems with varying capabilities.

• Create tools to validate the accuracy and latency of deployed models at scale with minimal friction.

• Recommend and implement model architecture changes to improve the accuracy-latency balance.

What we need to see:

• MSc or PhD in Computer Science, Engineering, or a related field, or equivalent professional experience.

• Over 5 years of confirmed experience specializing in model inference and optimization.

• 10+ years of work experience in a relevant area

• Expertise in modern machine learning frameworks, particularly PyTorch, ONNX, and TensorRT.

• Proven experience in optimizing inference for transformer and convolutional architectures.

• Strong programming proficiency in CUDA, Python, and C++.

• In-depth knowledge of optimization techniques, including quantization, pruning, distillation, and hardware-aware neural architecture search.

• Skilled in building and deploying scalable, cloud-based inference systems.

• Passionate about developing efficient, production-ready solutions with a strong focus on code quality and performance.

• Meticulous attention to detail, ensuring precision and reliability in safety-critical systems.

• Strong collaboration and communication skills for working optimally across multidisciplinary teams.

• A proactive, diligent mentality with a drive to tackle complex optimization challenges.

Ways to stand out from the crowd:

• Publications or industry experience in optimizing and deploying model inference at scale.

• Hands-on expertise in hardware-aware optimizations and accelerators such as GPUs, TPUs, or custom ASICs.

• Active contributions to open-source projects focused on inference optimization or machine learning frameworks.

• Experience in designing and deploying inference pipelines for real-time or autonomous systems.

The base salary range is 220,000 USD - 339,250 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.