Sr. AI Inference Systems Engineer
US-California-Palo Alto, United States
USD 120K-225K Senior-level Full Time
Tasks
- Build high-performance inference frameworks
- Design kv cache storage strategies
- Design router architecture
- Design technical roadmaps
- Develop standardized inference optimization schemes
- Evaluate inference architectures for real time batch and streaming
- Lead inference optimization technical bottleneck resolution
- Mentor team members
- Optimize inference operators for throughput and latency
- Optimize inference pipeline for large models
- Optimize scheduling and memory management
- Productize emerging inference technologies
- Research hardware accelerator inference logic
- Resolve distributed inference communication latency
- Resolve load imbalance in distributed inference
- Track compiler optimization model compression hardware fusion
Perks/Benefits
- 401k
- Dental insurance
- Disability insurance
- Health insurance
- Life insurance
- Paid Holidays
- Paid sick leave
- Paid vacation
- Relocation assistance
- Restricted stock units
- Sign-on bonus
- Vision insurance
Skills/Tech-stack
CUDA | Distributed Systems | Hardware Accelerators | Inference Optimization | Instruction set | Instruction set architecture | Intelligent routing | KV cache | Language Models | Large Language Models | Memory Management | Model Compression | Multimodal Models | Parallel Computing | PyTorch | Quantization | Router architecture | Scheduling | TensorFlow | Triton
Education
Regions
Countries
States
Cities
Related jobs
-
Amazon S3 | Data Engineering | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeNew York11h ago
-
Amazon S3 | Automation | Data Engineering | Data Modeling | Data Pipelines401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimePrinceton11h ago
-
Senior Databricks Forward Deployed Engineer - GPS USD 119K-198KAPI Integration | AWS | Airflow | Azure | CI/CDTravelSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Atlanta, Georgia, …11h ago
-
Lead AI and Data Solutions Engineer II USD 137K-229KAmazon Web Services | Apache Spark | Application Programming | Application Programming Interfaces | Cloud ComputingSenior-level Full TimeSacramento, California, United States; Tempe, Arizona, …11h ago
-
Software Engineer, Systems ML - SW/HW Co-design USD 117K-173KAI infrastructure | Bias Mitigation | C# | C++ | Co-designSenior-level Full TimeSunnyvale, CA | Redmond, WA12h ago
-
Software Engineer, Machine Learning USD 213K-293KAPI Design | Agent Orchestration | Artificial Intelligence | Bias Mitigation | C++Senior-level Full TimeSunnyvale, CA | Remote, US | … R12h ago
-
Senior Software Engineer, Generative AI, Google Ads USD 174K-252KComputer Vision | Data Processing | Debugging | GenAI | Information RetrievalSenior-level Full TimeMountain View, CA, USA12h ago
-
Staff Software Engineer, AI/ML Performance USD 207K-300KAlgorithms | Auto sharding | C++ | Code debugging | Code generationSenior-level Full TimeSunnyvale, CA, USA13h ago
-
Senior Software Engineer, Generative AI USD 174K-252KAgent-based | Agent-based systems | Cloud platform | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA13h ago
-
Software Engineer III, Generative AI, Payments Risk USD 147K-211KAgent systems | Algorithms | Analytics | Big Data | Computer VisionSenior-level Full TimeMountain View, CA, USA13h ago
-
Software Engineer III, Infrastructure, Infra Spanner USD 147K-211KC++ | Concurrency | Consensus Algorithms | Data Corruption | Data corruption diagnosisSenior-level Full TimeSunnyvale, CA, USA13h ago
-
C++ | Data Analysis | Data Processing | Deep learning | EmbeddingsSenior-level Full TimeMountain View, CA, USA13h ago
-
Apache Flume | C++ | Data Modeling | Data Processing | Data StructuresSenior-level Full TimeMountain View, CA, USA13h ago
-
Machine Learning Research Engineer USD 146K-222KData Analysis | Data Visualization | Deep learning | GPU Programming | Graph Neural Networks401k | Education reimbursement program | Flexible benefits package | Flexible schedule | Relocation assistanceMid-level Full TimeLivermore, CA, United States19h ago
-
Senior Machine Learning Engineer USD 229K-360KAB Testing | AWS SageMaker | Airflow | Amazon S3 | Apache FlinkDisability benefits | Equity awards | Health insurance | Life insurance | Paid time offSenior-level Full TimeSan Jose, California20h ago
-
Member of Technical Staff, Robotics Research Engineer USD 270K-370KData collection | Deep learning | Demonstration data | Diffusion Models | JAXSenior-level Full TimeNew York22h ago
-
Software Engineer- BIS (Baseten Inference Stack) USD 180K-360KAutoscaling | Backend Engineering | Distributed Runtime | Distributed Systems | GPU WorkloadsCompany 401K | Family building stipend | Flexible PTO | Medical/Dental/Vision insurance | Paid parental leaveSenior-level Full TimeSan Francisco23h ago
-
Sr. Software Development Engineer, Aurora Storage USD 168K-227KAWS | Amazon Aurora | Cross Region | Cross-region replication | Distributed SystemsLearning opportunities | Mentorship | Work-life balanceSenior-level Full TimeRedmond, Washington, USA1d ago
-
Sr. Software Development Engineer, Aurora Storage USD 168K-227KAWS | Auto Scaling | Auto-healing | Cross Region | Cross-region replicationCareer growth resources | Flexible work | Mentorship | Work-life balanceSenior-level Full TimeRedmond, Washington, USA1d ago
-
Gen AI Engineering Analyst - Vice President USD 113K-170KAWS | Accuracy | Apache Kafka | Apache Spark | Azure401k | Accident insurance | Disability insurance | Life insurance | Medical, dental, and vision coverageExecutive-level Full Time14000 CITI CARDS WAY BUILDING C …1d ago
-
Principal AI/ML Engineer USD 165K-226KC# | C++ | CI/CD | CUDA | Computer Vision401k match | Dental insurance | Health insurance | Life insurance | Paid time offSenior-level Full TimeRemote PA - PA PAR, United … R1d ago
-
APIs | Compliance | Distributed Systems | Enterprise Integration | Generative AIOccasional evening calls | Remote workSenior-level Full TimeRemote - US Based R1d ago
-
Senior AI Engineer USD 74K-147KAI Builder | API Development | AWS | Azure | Azure MLFlexible remote work policy | Flexible work-life balance | Knowledge sharing | Professional development | Supportive environmentSenior-level Full TimeChicago, United States1d ago
-
Senior Agentic AI Engineer USD 83K-203KArtificial Intelligence | Azure OpenAI | Cloud Computing | Code review | Data PipelinesDental insurance | Medical insurance | Paid time off | Retirement savings options | Vision insuranceSenior-level Full TimeWork At Home-Texas, United States1d ago
-
Machine Learning Research Engineer USD 99K-225KBenchmarking | Code review | Computer Vision | Conformal Prediction | Contrastive LearningPaid leave | Professional development | Tuition assistanceMid-level Full TimeUSA, VA, Springfield (7500 Geoint Dr), …1d ago