Software Engineer, LLM Infrastructure
Tasks
- Build Jupyter notebook tools for emulated and physical systems
- Build continuous batching for LLM serving
- Cache common pre fills
- Create interactive documentation for customers
- Develop algorithms for balancing prefill and completion tokens
- Develop speculative decoding and KV cache management
- Improve TTFT and end to end latency
- Port software from pre silicon to physical chips
- Profile and reduce network latency
Perks/Benefits
Skills/Tech-stack
Continuous batching | Jupyter | KV cache | Low Latency | Machine Learning | Network Latency | Profiling | Python | Speculative decoding | Token Caching | Transformer
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
AI Agents | AI Safety | AI Search | AWS | Agentic Workflows401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States6h ago
-
Hiring for Agentic Software Engineer | C2C | 12+ Months USD 139K-196KAPI Design | Amazon Q | Amazon Q Developer | Automated testing | C#Senior-level Contract Full TimeTexas City, TX, United States8h ago
-
Active Learning | Deep learning | Fine Tuning | Golang | Human FeedbackRemote work flexibility | Workplace accommodation supportSenior-level Full TimeMountain View, CALIFORNIA, United States10h ago
-
C++ | Distributed Training | ETL | Go | Hugging FaceInclusive work environment | Remote work flexibilitySenior-level Full TimeMountain View, CALIFORNIA, United States12h ago
-
Data Scientist, AI/ML – Visa Consulting and Analytics USD 123K-191KApache Spark | Embeddings | Excel | GenAI | GitHub Copilot401 K | Dental insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeAshburn, VA, United States12h ago
-
Data Engineer (remote) USD 87K-110KAgile | BigQuery | Data Governance | Data Modeling | Data Pipelines401k match | Employee assistance program | Employee stock purchase plan | Flexible schedule | Health insuranceMid-level Full TimeWork From Home, United States R12h ago
-
Forward Deployed Solution Engineer- Applied AI USD 201K-352KAWS | Angular | Azure | CI/CD | Context Management401k match | ESPP | Family Leave Program | Flexible time away | Flexible work arrangementsSenior-level Full TimeSanta Clara, California, United States13h ago
-
Senior-level Full TimeAnnapolis Junction, MD14h ago
-
AI Engineer (React UI) - Remote US USD 135K-195KAWS | Accessibility (WCAG) | Anthropic Claude | Apache Airflow | AzureHealth insurance | Paid time off | Remote workSenior-level Full TimeWauwatosa, WI, United States R14h ago
-
Senior-level Full TimeFort Lee, New Jersey, United States14h ago
-
AWS | Azure | CI/CD | Data Science | Docker401-k match | Dental insurance | Disability insurance | Life insurance | Medical coverageMid-level Full TimeHouston, TX, United States15h ago
-
Azure/AI DevOps Engineer III USD 102K-234KAI Search | AKS | AWS | Application Gateway | Azure AI401K Retirement Plan Matching | Fertility adoption and surrogacy support | Learning and development opportunities | Medical, dental, and vision coverage | Mental health supportSenior-level Full TimeRemote, United States R15h ago
-
Federal AI Solutions Engineer USD 135K-150KAKS | AWS | AWS Bedrock | AWS CDK | AWS Cloud401k match | Dental insurance | Health insurance | Paid time off | Professional development opportunitiesMid-level Full TimeContinental United States16h ago
-
Artificial Intelligence/Machine Learning Engineer USD 130K-177KData Modeling | Deep learning | Generative AI | Language Processing | Load forecasting401k match | Continuing education | Extra vacation days | Fitness device reimbursement | Flex TimeSenior-level Full TimeIndianapolis, IN, United States16h ago
-
AWS | Agentic AI | Angular | CI/CD | DatabricksHybrid work | Technical mentorshipSenior-level Full TimeNormal, United States17h ago
-
Sr. Data Engineer USD 108K-158KAWS | Apache Spark | Automated testing | Azure Event | Azure Event Hubs401k matching | Dental insurance | Disability insurance | Educational growth | Employee discount programSenior-level Full TimeNew York-TONAWANDA17h ago
-
Research Scientist - LLM Training System as a Service - Global Frontier Tech Recruitment Program - 2027 Start (PhD) USD 212K-450KCUDA | Deep learning | Distributed Systems | GPU Performance | GPU Performance OptimizationEntry-level Full TimeSan Jose, California, United States17h ago
-
Artificial Intelligence | Data Modeling | Data Pipelines | Data Quality | Data Visualization401k match | Dental insurance | Life insurance | Medical insurance | Paid time offSenior-level Full TimeNew York17h ago
-
Computer Vision | Distributed Training | Language Processing | Learning operations | Low LatencySenior-level Full TimeSan Jose, California, United States18h ago
-
Computer Vision | Deep learning | Language Processing | Machine Learning | ModelingEntry-level Full TimeSan Jose, California, United States18h ago
-
Data-Driven Decision Making | Data-driven | Decision Making | Deep learning | Distributed TrainingSenior-level Full TimeSunnyvale, CA19h ago
-
Production Engineer USD 178K-200KApache | Apache Spark | Application Programming | Application Programming Interfaces | C++Entry-level Full TimeMenlo Park, CA19h ago
-
Research Engineer - MSL FAIR Foundations USD 117K-173KBenchmarking | Code review | Data Pipelines | Distributed Systems | Language ModelEntry-level Full TimeMenlo Park, CA19h ago
-
AI Model Serving | AI model | Benchmarking | Cache Management | Data AnalysisSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA19h ago
-
Staff Software Engineer, AI Data Generation Platform USD 207K-300KComputer Vision | Data Engineering | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA19h ago