Senior Inference Engineer, AIConfigurator for Dynamo
US, CA, Santa Clara, United States
USD 184K-356K Senior-level Full Time
Tasks
- Build LLM serving optimization engine
- Build Python and Rust APIs
- Create CLIs and SDK surfaces
- Develop configuration search and SLA aware ranking
- Develop schema tests documentation and automation
- Estimate efficiency and latency
- Generate backend specific deployment artifacts
- Implement inference runtime abstractions for parallelism and batching
- Integrate performance databases and profiling data
- Perform Pareto frontier analysis
- Validate simulated performance against deployed results
Perks/Benefits
Skills/Tech-stack
Batching | Distributed Systems | Expert parallelism | GPU Computing | High Performance | High-Performance Computing | Inference Server | KV cache | Kubernetes | LLM Inference | Latency Estimation | Machine Learning | Machine Learning Infrastructure | Memory Management | NCCL | NIXL | NVSHMEM | Performance Computing | Pipeline parallelism | Prefill Decode | Prefill decode disaggregation | Python | Rust | SGLang | Tensor Parallelism | TensorRT-LLM | Triton Inference | Triton Inference Server | VLLM
Education
Regions
Countries
States
Cities
Related jobs
-
Perception Engineer USD 170K-275K2D vision | 3D Vision | C plus plus | Camera Calibration | Computer Vision401k | Dental insurance | End-of-year shutdown | Medical insurance | Mental health supportSenior-level Full TimeNew York, NY9h ago
-
Security Engineer (Embedded & Networking) USD 130K-175KApplication Firewall | C# | C++ | Cloud Security | GoExtended hours weekends as needed | Onsite work requirementMid-level Full TimeCape Canaveral, FL10h ago
-
Security Engineer (Embedded & Networking) USD 130K-180KAPI | Access Control | Access Management | Application Firewall | Authentication401k plan | Dental coverage | Disability insurance | Employee stock purchase plan | Life insuranceMid-level Full TimeHawthorne, CA10h ago
-
Senior Quantitative Developer USD 165K-240KAWS | Alerting | Amazon Kinesis | Cloud platform | Data Transformation401k match | Dental insurance | Hardware provided | Health insurance | Unlimited PTOSenior-level Full TimeNew York10h ago
-
Security Engineer (Embedded OT) USD 130K-180KAccess Control | Access Management | Application Firewall | Automation | Best practices401k retirement plan | Employee stock purchase plan | Health, dental, and vision insurance | Life insurance | Long-term disability insuranceMid-level Full TimeCape Canaveral, FL10h ago
-
Security Engineer (Embedded OT) USD 130K-180KAccess Control | Application Firewall | Authentication | Authorization | C#401k plan | Dental coverage | Disability insurance | Life insurance | Medical coverageMid-level Full TimeHawthorne, CA10h ago
-
Senior Data Engineer USD 140K-180KAccess Control | Alerting | Audit Logging | Cloud Data | Cloud Data WarehouseDental insurance | Fitness stipend | Health insurance | Meals covered | Paid time offSenior-level Full TimeMcLean, VA11h ago
-
C++ | Cloud Computing | Code Reviews | Deployment Automation | Distributed Systems401k match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R11h ago
-
Machine Learning Operations Engineer USD 133K-167KAWS SageMaker | Docker | GitHub Actions | Machine Learning | NumPyCareer development | Communities | Commuting cost coverage | Corporate giving programs | Daily free lunchMid-level Full TimeBoston, Massachusetts, United States R12h ago
-
Senior Software Engineer, Robotic Solutions Development USD 174K-237KBazel | C++ | CAD | GRPC | GazeboSenior-level Full TimeMountain View, California13h ago
-
Data Engineer Associate USD 75K-85KASM | Amazon Web Services | Azure | Cloud platform | Container Orchestration401k matching | Dental insurance | Health insurance | Life insurance | On-call supportEntry-level Full TimeRoseville, CA, United States13h ago
-
Senior Machine Learning Engineer USD 188K-282KAdversarial Training | Calibration monitoring | Continuous batching | DPO | Deep learningSenior-level Full TimePalo Alto, CA13h ago
-
Senior-level Full TimeRemote - United States R13h ago
-
Analytics Engineer - Music Creator Growth USD 116K-167KAmazon Redshift | BigQuery | Cloud Data | Cloud Data Platforms | DBT401k retirement plan | Flexible holidays | Health insurance | Meal allowance | Paid parental leaveMid-level Full TimeNew York, NY13h ago
-
Senior ML Engineer, Manipulation USD 150K-265KComputer Vision | Data Augmentation | Data pipeline | Dataset curation | Diffusion ModelsOnsite five days a weekSenior-level Full TimeSan Francisco14h ago
-
Senior ML Engineer, Foundation Models USD 150K-260KBehavior Cloning | Data Augmentation | DeepSpeed | Diffusion Models | Distributed TrainingHigh ownership team | Onsite work five days per week | Startup environmentSenior-level Full TimeSan Francisco14h ago
-
Anomaly Detection | Automated testing | CI/CD | Confluence | Data ModelingSenior-level Full TimeLos Angeles; New York; San Francisco15h ago
-
AI/ML Software Engineer USD 83K-83KAWS | Angular | Azure | CI/CD | Deep learning401k matching | Flex PTO | Hybrid work | Life insurance | Medical/Dental/Vision insuranceMid-level Full TimeBay Area, CA15h ago
-
NYC Sr. AI Engineer: Assets, Formats & Placements USD 139K-229KA/B | A/B Testing | Apache Lucene | Apache Solr | B testingFlexible work hybrid option | Health and wellness programs | Time away from workSenior-level Full TimeNew York, NY, United States16h ago
-
AI Developer Experience Engineer USD 180K-250KAPI Development | Anthropic | Automation | Benchmarking | Data PipelinesSenior-level Full TimeNew York City16h ago
-
Forward Deployed AI Engineer, West USD 125K-175KAWS | Azure | Docker | GCP | Generative AI401k plan | Dental insurance | Medical insurance | Parental leave | Unlimited paid time offMid-level Full TimeRemote (San Francisco) R16h ago
-
Senior Machine Learning Engineer, Reinforcement Learning USD 150K-250KDomain Randomization | Embedded Systems | Gazebo | Isaac-Gym | Mujoco401k retirement plan | Dental insurance | Employee referral bonus | Flexible PTO | Free lunchSenior-level Full TimeColumbus, Ohio or Remote R16h ago
-
Senior Data Platform Engineer USD 140K-220KApache Hudi | Apache Spark | CI/CD | Delta Lake | Distributed StorageSenior-level Full TimePittsburgh, PA or Remote R17h ago
-
Senior AI Engineer, Ads Ai USD 139K-229KA I | A I Data Science | A/B | A/B Testing | B testingFlexible work | Health and wellness programs | Time away from workSenior-level Full TimeNew York, NY, United States17h ago
-
Senior Software Engineer - Analytics Front Office USD 200K-250KAlgorithms | Asynchronous programming | C# | C++ | CI/CD401k | Dental insurance | Disability insurance | Flexible spending account | Health insuranceSenior-level Full TimeChicago, Greenwich, New York, Austin, Houston17h ago