Sr. AI Inference Systems Engineer
US-California-Palo Alto, United States
USD 120K-225K Senior-level Full Time
Tasks
- Build high-performance inference frameworks
- Design kv cache storage strategies
- Design router architecture
- Design technical roadmaps
- Develop standardized inference optimization schemes
- Evaluate inference architectures for real time batch and streaming
- Lead inference optimization technical bottleneck resolution
- Mentor team members
- Optimize inference operators for throughput and latency
- Optimize inference pipeline for large models
- Optimize scheduling and memory management
- Productize emerging inference technologies
- Research hardware accelerator inference logic
- Resolve distributed inference communication latency
- Resolve load imbalance in distributed inference
- Track compiler optimization model compression hardware fusion
Perks/Benefits
- 401k
- Dental insurance
- Disability insurance
- Health insurance
- Life insurance
- Paid Holidays
- Paid sick leave
- Paid vacation
- Relocation assistance
- Restricted stock units
- Sign-on bonus
- Vision insurance
Skills/Tech-stack
CUDA | Distributed Systems | Hardware Accelerators | Inference Optimization | Instruction set | Instruction set architecture | Intelligent routing | KV cache | Language Models | Large Language Models | Memory Management | Model Compression | Multimodal Models | Parallel Computing | PyTorch | Quantization | Router architecture | Scheduling | TensorFlow | Triton
Education
Regions
Countries
States
Cities
Related jobs
-
Mid-Level Data Engineer USD 90K-98KAPI Development | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageRemote workMid-level Full TimeWork from home, VA, United States R6h ago
-
Senior Data Engineer USD 165K-180KAPIs | Anomaly Detection | Azure | Azure Data | Azure Data FactorySenior-level Full TimeWork from home, VA, United States R6h ago
-
Evergreen - Mathematics for Machine Learning USD 80K-300KAutodiff | JAX | Linear Algebra | Matrix Operations | NumPyAsynchronous hiring process | Flexible collaboration | Part-time hoursMid-level Full TimeBoston, US9h ago
-
Computer Vision | Data Analysis | Language Models | Language Processing | Large Language ModelsSenior-level Full TimeSeattle, Washington, United States10h ago
-
Classification Algorithms | Data Analysis | Deep learning | Language Models | Language ProcessingSenior-level Full TimeSan Jose, California, United States10h ago
-
Algorithms | Audio Software | C++ | Debugging | Embedded SystemsSenior-level Full TimeMountain View, CA, USA11h ago
-
Software Engineer, Machine Learning USD 207K-300KC++ | Data Processing | Experimentation | Information Retrieval | Just-in-TimeSenior-level Full TimeNew York, NY, USA; Mountain View, …11h ago
-
Customer Engineer, Data Analytics, Google Cloud USD 153K-222KBatch Processing | Big Data | Cloud Architecture | Cloud platform | Customer RequirementsSenior-level Full TimeSunnyvale, CA, USA11h ago
-
C++ | Data Processing | Debugging | Information Retrieval | Language ModelsSenior-level Full TimeMountain View, CA, USA11h ago
-
Algorithms | C++ | Cloud Computing | Cloud platform | Data StructuresSenior-level Full TimeSunnyvale, CA, USA11h ago
-
Cloud Data and AI Engineer, Professional Services USD 127K-183KC++ | Capacity Planning | Cloud Databases | Data Migration | Data PipelinesTravel up to 30 percentMid-level Full TimeReston, VA, USA11h ago
-
Staff Software Engineer, ML Frameworks USD 207K-300KAPIs | Data Processing | Debugging | Fine Tuning | GPU AccelerationSenior-level Full TimeMountain View, CA, USA11h ago
-
AWS | Artificial Intelligence | Azure AI | Data Analysis | DatabricksBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeChicago, IL, United States21h ago
-
Staff Machine Learning Engineer (Pricing) USD 215K-322KA/B Testing | API Design | AWS | Automated retraining | B testing401k matching | Dental insurance | Family planning assistance | Flexible time off | Healthcare benefitsSenior-level Full TimeSan Francisco, CA22h ago
-
Data Platform Engineer USD 205K-316KApache Airflow | Automation | Cloud infrastructure | Cloud platform | DBT401k match | Commuter benefits | Dental insurance | Employee stock options | Health insuranceMid-level Full TimeDenver, CO; New York, NY; San …22h ago
-
Anomaly Detection | Calibration | Classification | Clustering | Decision TreesCommuter benefits | Disability benefits | Health insurance | Life insurance | Paid time offSenior-level Full TimeNew York, New York22h ago
-
Full-Stack AI Software Engineer USD 120K-150KAWS | Agile | Algorithms | Azure | C#401k company match | Company holidays | Deferred compensation plan | Dental insurance | Disability insuranceMid-level Full TimeCRC - Alpharetta, GA 3460 Preston …22h ago
-
Senior Staff Data Engineer USD 155K-262KAWS | Airflow | Apache Flink | Apache Spark | Azure401k match | Accidental death and dismemberment | Dental insurance | Employee assistance program | Equity eligibilitySenior-level Full TimeUnited States, United States22h ago
-
AWS | C++ | EC2 | Git | GoSenior-level Full TimeNew York, United States22h ago
-
Data Domain Architect Lead USD 171K-205KArtificial Intelligence | Business Intelligence | Data Annotation | Data Modeling | Data ProcessingBackup childcare | Financial coaching | Health care coverage | Mental health support | On Site Health Wellness CentersSenior-level Full TimeWilmington, DE, United States1d ago
-
Senior Staff Machine Learning Engineer USD 173K-303KAnsible | Cloud Native | Communication Systems | Configuration Management | DevOpsSenior-level Full TimeHartford, Connecticut, United States1d ago
-
AI Engineer USD 139K-198KAI Search | AKS | AWS Bedrock | Amazon SageMaker | AutogenLeadership development | Professional developmentSenior-level Full TimeWashington, DC1d ago
-
Software Engineer, Hardware Health USD 250K-445KAutomated remediation | Distributed Systems | Fleet Lifecycle Management | Infiniband | Infrastructure PlatformsSenior-level Full TimeSan Francisco1d ago
-
AI Agents | Apache Spark | Data Ingestion | Data Modeling | Data Transformation401k match | Company provided disability insurance | Dental insurance | Flexible spending accounts | Health care and dependent care flexible spending accountsSenior-level Full TimeUnited States1d ago
-
AI Solutions Engineer, East USD 125K-175KAWS | Azure | Cloud platform | Dspy | Generative AI401k plan | Dental insurance | Medical insurance | Mental wellness support | Parental leaveMid-level Full TimeRemote (New York) R1d ago