Distinguished Engineer, Scaled Out Inferencing
Tasks
- Architect distributed inference pipelines
- Automate deployment and scaling
- Build systems and orchestration layers for model serving
- Collaborate on hardware-software co-optimization
- Contribute to open-source inference projects
- Coordinate with customers infrastructure providers and partners
- Define technical roadmap for model serving systems
- Design high throughput low latency distributed inference
- Implement kernel and driver level performance tuning
- Lead full lifecycle model deployment and versioning
- Optimize GPU resource management and hardware acceleration
Perks/Benefits
- N/A
Skills/Tech-stack
AI infrastructure | CUDA | Cloud Native | Cloud Native Architecture | Container Orchestration | Distributed Systems | GPU Architecture | Hardware acceleration | High Throughput | High-Throughput Systems | Kernel programming | Kubernetes | Linux | Low Latency | Low-Latency Systems | Model Serving | Multi-tenant | Multi-tenant architecture | Ray | SGLang | Tenant architecture | TensorRT-LLM | VLLM
Education
Roles
Regions
Countries
States
Related jobs
-
Senior AI Engineer, High Performance AI USD 152K-241KAgent systems | C# | C++ | CUDA | Compiler optimizationSenior-level Full TimeUS, CA, Santa Clara R5d ago
-
Checkpointing | Data Pipelines | Data Processing | Data loading | Data parallelismSenior-level Full TimeUS, CA, Santa Clara R6d ago
-
Senior-level Full TimeUS, CA, Santa Clara R8d ago
-
Architecture Search | CUDA | CUDA Compilation | CUDA kernel | GPU ProfilingSenior-level Full TimeUS, CA, Santa Clara R9d ago
-
Senior Solutions Architect, Inference Service Providers USD 184K-287KAgent Design | Cache Management | Custom Inference Kernels | Data Curation | DevOpsSenior-level Full TimeUS, CA, Santa Clara R12d ago
-
Array computing | Asynchronous programming | C++ | CUDA | Compiler optimizationBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R12d ago
-
Senior Software Engineer, RL Post-Training Frameworks USD 184K-287KActor Based Programming | C# | C++ | Deep learning | Distributed SystemsSenior-level Full TimeUS, CA, Santa Clara R12d ago
-
Senior-level Full TimeUS, CA, Santa Clara R13d ago
-
Senior AI Infrastructure Software Engineer - DGX Cloud USD 184K-287KC# | C++ | Cause analysis | Debugging | Distributed SystemsBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R13d ago
-
Senior-level Full TimeUS, CA, Santa Clara R14d ago
-
Senior Deep Learning Algorithm Engineer USD 184K-287KCPU architecture | Distributed Computing | GPU Architecture | Generative AI | JAXEquity | Health benefitsSenior-level Full TimeUS, CA, Santa Clara R19d ago
-
Senior Deep Learning Compiler Engineer - XLA USD 152K-241KC# | C++ | CUDA | Code generation | DebuggingSenior-level Full TimeUS, CA, Santa Clara R19d ago
-
C++ | CUDA | CuDF | DataFrame | Database accelerationSenior-level Full TimeUS, CA, Santa Clara R19d ago
-
C++ | CUDA | Docker | Infiniband | JAXSenior-level Full TimeUS, CA, Santa Clara R19d ago
-
Software Engineer, CUDA Deep Learning Systems USD 152K-258KC++ | CUDA | Computer Architecture | Deep learning | Distributed ComputingBenefits | EquityMid-level Full TimeUS, CA, Santa Clara R22d ago
-
C# | C++ | Caching | Distributed Systems | GPU ArchitectureComprehensive benefits | EquitySenior-level Full TimeUS, CA, Santa Clara R27d ago
-
Senior Software Engineer, CUDA Deep Learning Systems USD 184K-356KC++ | CUDA | Computer Architecture | Deep Learning Compilers | Deep learningBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R28d ago
-
Senior Storage Production Engineer - DGX Cloud USD 184K-333KAccess Control | Anomaly Detection | Ansible | Auditing | BashSenior-level Full TimeUS, CA, Santa Clara R28d ago
-
C++ | CPU architecture | CUDA | Deep learning | Fraud DetectionSenior-level Full TimeUS, CA, Santa Clara R28d ago
-
Bash | Bootstrap | CSI | CSS3 | Container StorageSenior-level Full TimeUS, CA, Santa Clara R29d ago
-
Senior Deep Learning Frameworks CUDA Software Engineer USD 184K-356KC++ | CUDA | Distributed machine learning | HPC | JAXSenior-level Full TimeUS, CA, Santa Clara R30d ago
-
Senior Deep Learning Framework Communications Engineer USD 184K-287KC++ | CUDA | CuTe | Elasticity | Fault ToleranceBenefits | EquitySenior-level Full TimeUS, CA, Santa Clara R30d ago
-
Senior Cloud Software Engineer, DGXC Data Services USD 152K-241KAPI Design | Algorithms | Amazon Web Services | C++ | CheckpointsEquity | Health benefitsSenior-level Full TimeUS, CA, Santa Clara R1mo ago
-
C# | C++ | CUDA | Communication Scaling | LinuxEquity | Health insurance | Paid time off | Retirement planSenior-level Full TimeUS, CA, Santa Clara R1mo ago
-
Asynchronous programming | Context Management | Deep learning | Distributed Systems | GPUSenior-level Full TimeUS, CA, Santa Clara R1mo ago