Application Software Engineer, Inference
Tasks
- Architect model serving infrastructure
- Benchmark and accelerate inference engines
- Build reliable high concurrency serving systems
- Collaborate to integrate inference into systems
- Create CI CD infrastructure
- Develop inference systems
- Develop tools for tracing and replay
- Optimize inference latency and throughput
- Own request routing and rate limiting
Perks/Benefits
- 401k plan
- Employee stock purchase plan
- Long-term incentives
- Medical, dental & vision coverage
- Onsite Palo Alto
- Paid Holidays
- Paid parental leave
- Paid vacation
Skills/Tech-stack
Agent Orchestration | Agent SDK | Auto Scaling | Batch scheduling | C++ | CI/CD | ClickHouse | Continuous batching | Continuous integration | Distributed Systems | Docker | GPU Kernel | GRPC | Global KV Cache | Go | Inference | KV cache | Kubernetes | Language Models | Large Language Models | Load Balancing | MongoDB | Observability | Performance optimization | PostgreSQL | Profiling | Python | Quantization | REST | Rust | Speculative decoding
Education
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R3d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R17d ago
-
AI | Adversarial Machine Learning | Bot Detection | Data Analysis | Data IntegrityMid-level Full TimeMountain View, CA, USA4h ago
-
Staff Software Engineer, AI/ML, Google Cloud Security USD 207K-301KAgentic AI | Cloud platform | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeReston, VA, USA; New York, NY, …4h ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R11h ago
-
Senior-level Full TimeMountain View, CA13h ago
-
Staff Software Engineer, AI Integrations USD 170K-240KAgile | Alerting | Amazon Web Services | C++ | CSSSenior-level Full TimeMountain View, CA13h ago
-
A B Updates | A/B | Bitbake | Buildroot | C#401k matching | Employee assistance program | FSA | HSA | Life insuranceEntry-level Full TimeSan Diego, CA14h ago
-
Perception Engineer III USD 155K-185KArduPilot | C++ | CUDA | Coordinate frames | DDS401k company match | Annual Company Holidays | Life insurance | Medical, dental & vision coverage | Paid time offSenior-level Full TimeUnited States18h ago
-
Senior Software Engineer Remote/Travel- Active Secret -Top Secret clearance(SCI-eligibility) USD 145K-204KAPI Testing | Cypher | Data Processing | DataOps | DevOpsCompetitive benefits | Growth opportunity | Remote work | TravelSenior-level Full TimeReston, VA, United States R23h ago
-
AWS | AWS CloudFormation | Amazon ECS | Automation | CI/CD401-k plan | Company match | Employee stock purchase plan | Healthcare benefits | Life insuranceMid-level Full TimeAtlanta, GA, United States1d ago
-
Machine Learning Engineer USD 124K-170KAWS Bedrock | AWS Fargate | AWS Lambda | AWS Step Functions | Amazon ECSCareer development opportunities | Equal employment opportunityMid-level Full TimePlano, Texas, United States1d ago
-
Senior Staff Generative AI Scientist USD 185K-210KAWS | Apache Spark | Artificial Intelligence | Azure | Cloud platform401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeRemote, United States R1d ago
-
Machine Learning Engineer USD 117K-177KAWS | Azure | Cloud Computing | Data Visualization | Google CloudMid-level Full TimeSt. Louis, Missouri, United States1d ago
-
Software Engineer - X Data USD 125K-400KApache Flink | Apache Kafka | Apache Spark | BigQuery | Cause analysis401k retirement plan | Dental insurance | Discounts | Health insurance | Life insuranceMid-level Full TimePalo Alto, CA1d ago
-
Embedded Software Engineer Intern (Fall 2026) USD 108K-108KADC | Antenna Testing | C# | C++ | CANHousing stipend | Overtime pay | Paid sick time | Relocation supportEntry-level InternshipSouth San Francisco, California, USA1d ago
-
Site Reliability Engineer, Inference Infrastructure USD 165K-267KAmazon Web Services | C plus plus | Cloud platform | Distributed Systems | GPU401k matching | Co working Benefit | Company offsite | Education and learning stipend | Health and dental benefitsMid-level Full TimeToronto1d ago
-
Staff Software Engineer, Inference Infrastructure USD 160K-220KAmazon Web Services | Azure | C++ | Cloud platform | Distributed Systems401k | Annual company offsite | Arts and culture budget | Education and learning stipend | Health and dental benefitsSenior-level Full TimeSan Francisco1d ago
-
AI Engineer – Decision & Optimization Systems USD 80K-210KAMPL | C++ | CPLEX | Constraint Satisfaction | Go401k | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Engineer - Allocation and Packing Systems USD 80K-210KA/B | A/B Testing | Approximation algorithms | Assignment Algorithms | B testing401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Engineer – Routing & Network Optimization USD 80K-210KA Star | A Star Algorithm | AMPL | Bellman Ford Algorithm | CPLEX401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Software Engineer USD 80K-210KAWS Kinesis | Apache Airflow | Apache Kafka | C# | C++401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
Deployment Strategist - North America USD 150K-245KAI Agent | AI Agent Frameworks | API Integration | Agent Frameworks | Agent OrchestrationAnnual company offsite | Annual travel stipend | Co-working stipend | Professional development stipendMid-level Full TimeUnited States R1d ago
-
Applied AI Engineer USD 204K-352KBackend Development | Cloud infrastructure | Deep learning | Docker | Fine TuningAnnual membership | Centralized coffee stipend | Flexible PTO | Health and wellness stipend | Health, dental, and vision insuranceSenior-level Full TimeNew York, NY (HQ)1d ago
-
ML Research Engineer - Hardware Codesign USD 185K-455KC++ | CUDA | Floating point | Floating point numerics | Functional simulationHybrid work schedule | Relocation assistanceSenior-level Full TimeSan Francisco1d ago