Software Engineer, LLM Infrastructure
Tasks
- Build Jupyter notebook tools for emulated and physical systems
- Build continuous batching for LLM serving
- Cache common pre fills
- Create interactive documentation for customers
- Develop algorithms for balancing prefill and completion tokens
- Develop speculative decoding and KV cache management
- Improve TTFT and end to end latency
- Port software from pre silicon to physical chips
- Profile and reduce network latency
Perks/Benefits
Skills/Tech-stack
Continuous batching | Jupyter | KV cache | Low Latency | Machine Learning | Network Latency | Profiling | Python | Speculative decoding | Token Caching | Transformer
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Junior Software Engineer USD 74K-105KAPI Development | API Integration | AWK | AWS IAM | Amazon Web ServicesAgile team environment | Fast-paced environment | Onsite work | Security clearance supportEntry-level Full TimeSpringfield, VA, United States1h ago
-
Machine Learning Engineer, TikTok BRIC Account Security USD 145K-250KBehavioral analytics | Correlation Analysis | Data Warehousing | Data correlation | Data correlation analysisEntry-level Full TimeSan Jose, California, United States6h ago
-
Senior Data Engineer USD 84K-149KAWS | Agile | Apache Airflow | Apache Kafka | Apache Spark401k employer match | Annual incentive program | Dental insurance | Disability insurance | Flexible time offSenior-level Full TimeLisle, IL, United States7h ago
-
Agent systems | C++ | Computer Vision | Data Processing | DebuggingSenior-level Full TimeSunnyvale, CA, USA7h ago
-
Software Engineer III, Generative AI, Payments Risk USD 147K-211KAgent systems | Big Data | Computer Vision | Data Processing | Data StructuresSenior-level Full TimeMountain View, CA, USA7h ago
-
Senior Software Engineer, Google Cloud Storage USD 174K-252KAs-a-Service | C++ | Chaos Engineering | Cloud Functions | Cloud StorageSenior-level Full TimeRaleigh, NC, USA; Durham, NC, USA7h ago
-
Robotics Software Engineer – Robot Integrations USD 70K-300KC++ | Computer Vision | Control Systems | Linux | Operating SystemHybrid or remote optionMid-level Full TimeIrvine, CA16h ago
-
Applied AI/ML - Senior Associate USD 175K-210KAgentic AI | Amazon Bedrock | Amazon SageMaker | Cloud deployment | ContainerizationBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeJersey City, NJ, United States17h ago
-
Senior Data Engineer USD 157K-212KAWS OpenSearch | Airflow | Argo | Database optimization | Docker401k match | Flex hours | Hybrid work arrangement | Paid time offSenior-level Full TimeUSA VA Crystal City - 2521 …19h ago
-
Artificial Intelligence | Computer Vision | Docker | Kubernetes | Language ProcessingHybrid office first work model | Performance based bonus opportunity | Relocation considerationMid-level Full TimeChicago, Illinois, United States19h ago
-
Engineering Lead Analyst - VP USD 125K-188KAI Automation | AI coding | AI coding assistant | CI/CD | Cloud401k | Medical/Dental/Vision | Paid time offSenior-level Full Time6400 LAS COLINAS BLVD IRVING, United …19h ago
-
Software Engineer III - Fullstack, Python, AWS, AI/ML USD 163K-185KAPI Gateway | AWS | Audit Logging | Azure | Blue/greenBackup childcare | Financial coaching | Health care coverage | Mental health support | Retirement savings planSenior-level Full TimePlano, TX, United States21h ago
-
Sr Data Engineer, Compliance Data Infrastructure USD 223K-268KAccess Control | Alibaba MaxCompute | Amazon Redshift | Apache Hadoop | Apache SparkEducation subsidy | Healthcare benefits | Learning and development programs | Team building programs | Wellness and meal allowancesSenior-level Full TimeUnited States (US)23h ago
-
Sr Machine Learning Engineer, Compliance USD 223K-268KAnomaly Detection | Batch data | Batch data pipelines | CI/CD | Data PipelinesComprehensive healthcare | Education subsidy | Learning and development support | Meal allowance | Wellness allowanceSenior-level Full TimeSan Jose, California, United States23h ago
-
Director, Compliance Data Science & AI USD 313K-375KAnomaly Detection | Data Engineering | Graph analytics | LLM | MLOpsComprehensive healthcare | Education subsidy | L D programs | Meal allowances | Team building programsExecutive-level Full TimeSan Jose, California, United States23h ago
-
Deep learning | Diffusion Models | Distributed Training | Flow Models | Generative AIComprehensive benefits | Equity | Real time impact on productsMid-level Full TimePalo Alto, CA1d ago
-
Attention | C++ | CUDA | FP16 | FP8Daily lunch and dinner | Housing subsidy | Medical, dental, and vision | Relocation supportMid-level Full TimeCupertino, CA1d ago
-
C++ | CI/CD | Cloud Computing | Containerization | Infrastructure EngineeringMid-level Full TimeCupertino, CA1d ago
-
Bundle adjustment | C++ | Control Theory | Linear systems | Numerical OptimizationSenior-level Full TimeOrange County, CA1d ago
-
AWS | Cloud platform | Computer Vision | Data Engineering | Data labelingCustomer-facing opportunities | International relocation support | Published research culture | Startup environment | Support for EB visasMid-level Full TimeSan Francisco, CA1d ago
-
AY2024-2025 #6042 Research Faculty in AI/ML - O'Donnell Data Science and Research Computing Institute USD 88K-108KC plus plus | C# | CUDA | Deep learning | Generative AITechnical mentorshipEntry-level Full TimeDallas, TX1d ago
-
Full Stack Engineer, AI systems USD 165K-225KAPI Architecture | Anthropic API | Docker | Kubernetes | Language ModelsSenior-level Full TimePalo Alto, United States1d ago
-
API Development | AWS | Azure | Cloud | ComplianceExecutive-level Full TimePhiladelphia, Pennsylvania, United States1d ago
-
Data Scientist / Data Engineer (TS/SCI with Poly) USD 140K-180KBig Data | Machine Learning | Python | R401k | Employee discount program | Flexible work schedule | Health savings account | Medical, dental, and vision coverageMid-level Full TimeAnnapolis Junction, MD, US1d ago
-
Senior Data Engineer - Fort Gordon, Georgia (APOGEE) USD 99K-148KC# | Data Governance | Data integration | GPU Computing | Gensim401k matching | Dental insurance | Disability coverage | Health insurance | Life insuranceSenior-level Full TimeFt. Gordon, US-GA, US1d ago