Senior Inference Engineer, AIConfigurator for Dynamo
US, CA, Santa Clara, United States
USD 184K-356K Senior-level Full Time
Tasks
- Build LLM serving optimization engine
- Build Python and Rust APIs
- Create CLIs and SDK surfaces
- Develop configuration search and SLA aware ranking
- Develop schema tests documentation and automation
- Estimate efficiency and latency
- Generate backend specific deployment artifacts
- Implement inference runtime abstractions for parallelism and batching
- Integrate performance databases and profiling data
- Perform Pareto frontier analysis
- Validate simulated performance against deployed results
Perks/Benefits
Skills/Tech-stack
Batching | Distributed Systems | Expert parallelism | GPU Computing | High Performance | High-Performance Computing | Inference Server | KV cache | Kubernetes | LLM Inference | Latency Estimation | Machine Learning | Machine Learning Infrastructure | Memory Management | NCCL | NIXL | NVSHMEM | Performance Computing | Pipeline parallelism | Prefill Decode | Prefill decode disaggregation | Python | Rust | SGLang | Tensor Parallelism | TensorRT-LLM | Triton Inference | Triton Inference Server | VLLM
Education
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R4d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R17d ago
-
Software Engineer, Scientific Computing USD 90K-160KCUDA | Debugging | Deployment | Numerical Methods | Performance optimizationEntry-level Full TimeRedwood City5h ago
-
Entry-level Full TimeRedwood City5h ago
-
AWS Glue | AWS IAM | AWS Kinesis | AWS Lambda | AWS Step Functions100 percent on siteSenior-level ContractMc Lean, United States6h ago
-
AI | Adversarial Machine Learning | Bot Detection | Data Analysis | Data IntegrityMid-level Full TimeMountain View, CA, USA9h ago
-
Staff Software Engineer, AI/ML, Google Cloud Security USD 207K-301KAgentic AI | Cloud platform | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeReston, VA, USA; New York, NY, …9h ago
-
Senior-level Full TimeMountain View, CA, USA9h ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R15h ago
-
Senior-level Full TimeMountain View, CA17h ago
-
Staff Software Engineer, AI Integrations USD 170K-240KAgile | Alerting | Amazon Web Services | C++ | CSSSenior-level Full TimeMountain View, CA17h ago
-
A B Updates | A/B | Bitbake | Buildroot | C#401k matching | Employee assistance program | FSA | HSA | Life insuranceEntry-level Full TimeSan Diego, CA19h ago
-
Senior Principal AI Engineer USD 152K-254KData Processing | Git | Hugging Face | Language Models | Language ProcessingAdoption Assistance | Dental vision prescription coverage | Disability benefits | Employee assistance program | Health benefitsSenior-level Full TimeCambridge, United States20h ago
-
Principal / Sr Principal Data Analytics Engineer USD 98K-184KAgile | CI/CD | Data Governance | Data Modeling | Data MonitoringCompany-Paid Holidays | Disability insurance | Health insurance | Hybrid work | Life insuranceSenior-level Full TimeFLME229, United States20h ago
-
Sr. Data Engineer USD 109K-145KAWS | Agile | CI/CD | Compliance | Data Governance401k matching | Employee assistance program | Flexible working hours | Learning platform access | Medical, dental, and vision insuranceSenior-level Full TimeRemote, TX, United States R20h ago
-
Senior Machine Learning Engineer USD 134K-168KData Drift | Databricks | Ethical AI | Feature Engineering | KubernetesSenior-level Full TimeNew York, NY, United States20h ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-204KAPI Testing | Cypher | Data Processing | DataOps | DevOpsCompetitive benefits | Growth opportunity | Remote work | TravelSenior-level Full TimeReston, VA, United States R1d ago
-
Senior Software Engineer - Database USD 165K-225KC# | C++ | Databases | Distributed Systems | Fault ToleranceSenior-level Full TimeNew York, NY, US1d ago
-
Senior Software Engineer - Database USD 165K-225KAlgorithms | C# | C++ | Concurrency | Data StructuresSenior-level Full TimeNew York, NY, US1d ago
-
AWS | AWS CloudFormation | Amazon ECS | Automation | CI/CD401-k plan | Company match | Employee stock purchase plan | Healthcare benefits | Life insuranceMid-level Full TimeAtlanta, GA, United States1d ago
-
Associate Software Engineer (Python + AWS) USD 143K-194KAWS Lambda | AWS S3 | AWS SNS | AWS SQS | Amazon Web ServicesHybrid work | Work authorization not requiring visa sponsorship | Work from home up to three days per weekMid-level Full TimeBaltimore, Maryland, United States1d ago
-
Machine Learning Engineer USD 124K-170KAWS Bedrock | AWS Fargate | AWS Lambda | AWS Step Functions | Amazon ECSCareer development opportunities | Equal employment opportunityMid-level Full TimePlano, Texas, United States1d ago
-
Senior Staff Generative AI Scientist USD 185K-210KAWS | Apache Spark | Artificial Intelligence | Azure | Cloud platform401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeRemote, United States R1d ago
-
Machine Learning Engineer USD 117K-177KAWS | Azure | Cloud Computing | Data Visualization | Google CloudMid-level Full TimeSt. Louis, Missouri, United States1d ago
-
Sr. Database Engineer (MongoDB) USD 127K-236KAWS | Agile | Confluence | Django | Docker401k contribution | Career development | Dental insurance | Health insurance | Holiday paySenior-level Full TimeLinthicum Heights, MD1d ago