Inference Software Engineer
Tasks
- Build scheduling logic for continuous batching and real time inference
- Collaborate with AI researchers and product teams
- Contribute to architecture and design of host software stack
- Implement distributed networking primitives for multi server inference
- Implement high performance modular code
- Implement inference time acceleration techniques
- Interface with firmware and drivers teams
Perks/Benefits
Skills/Tech-stack
C++ | CUDA | Continuous batching | Distributed Systems | KV cache | Networking | Parallel Programming | Python | Rust | Speculative decoding | Transformer | Tree search
Education
N/A
Regions
Countries
States
Cities
Related jobs
-
AI Agents | AI Safety | AI Search | AWS | Agentic Workflows401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States6h ago
-
Hiring for Agentic Software Engineer | C2C | 12+ Months USD 139K-196KAPI Design | Amazon Q | Amazon Q Developer | Automated testing | C#Senior-level Contract Full TimeTexas City, TX, United States8h ago
-
Active Learning | Deep learning | Fine Tuning | Golang | Human FeedbackRemote work flexibility | Workplace accommodation supportSenior-level Full TimeMountain View, CALIFORNIA, United States10h ago
-
C++ | Distributed Training | ETL | Go | Hugging FaceInclusive work environment | Remote work flexibilitySenior-level Full TimeMountain View, CALIFORNIA, United States12h ago
-
Data Scientist, AI/ML – Visa Consulting and Analytics USD 123K-191KApache Spark | Embeddings | Excel | GenAI | GitHub Copilot401 K | Dental insurance | Health insurance | Life insurance | Paid time offMid-level Full TimeAshburn, VA, United States12h ago
-
Data Engineer (remote) USD 87K-110KAgile | BigQuery | Data Governance | Data Modeling | Data Pipelines401k match | Employee assistance program | Employee stock purchase plan | Flexible schedule | Health insuranceMid-level Full TimeWork From Home, United States R12h ago
-
Forward Deployed Solution Engineer- Applied AI USD 201K-352KAWS | Angular | Azure | CI/CD | Context Management401k match | ESPP | Family Leave Program | Flexible time away | Flexible work arrangementsSenior-level Full TimeSanta Clara, California, United States13h ago
-
Senior-level Full TimeAnnapolis Junction, MD14h ago
-
AI Engineer (React UI) - Remote US USD 135K-195KAWS | Accessibility (WCAG) | Anthropic Claude | Apache Airflow | AzureHealth insurance | Paid time off | Remote workSenior-level Full TimeWauwatosa, WI, United States R14h ago
-
Senior-level Full TimeFort Lee, New Jersey, United States14h ago
-
AWS | Azure | CI/CD | Data Science | Docker401-k match | Dental insurance | Disability insurance | Life insurance | Medical coverageMid-level Full TimeHouston, TX, United States15h ago
-
Azure/AI DevOps Engineer III USD 102K-234KAI Search | AKS | AWS | Application Gateway | Azure AI401K Retirement Plan Matching | Fertility adoption and surrogacy support | Learning and development opportunities | Medical, dental, and vision coverage | Mental health supportSenior-level Full TimeRemote, United States R15h ago
-
Federal AI Solutions Engineer USD 135K-150KAKS | AWS | AWS Bedrock | AWS CDK | AWS Cloud401k match | Dental insurance | Health insurance | Paid time off | Professional development opportunitiesMid-level Full TimeContinental United States16h ago
-
Artificial Intelligence/Machine Learning Engineer USD 130K-177KData Modeling | Deep learning | Generative AI | Language Processing | Load forecasting401k match | Continuing education | Extra vacation days | Fitness device reimbursement | Flex TimeSenior-level Full TimeIndianapolis, IN, United States16h ago
-
AWS | Agentic AI | Angular | CI/CD | DatabricksHybrid work | Technical mentorshipSenior-level Full TimeNormal, United States17h ago
-
Sr. Data Engineer USD 108K-158KAWS | Apache Spark | Automated testing | Azure Event | Azure Event Hubs401k matching | Dental insurance | Disability insurance | Educational growth | Employee discount programSenior-level Full TimeNew York-TONAWANDA17h ago
-
Research Scientist - LLM Training System as a Service - Global Frontier Tech Recruitment Program - 2027 Start (PhD) USD 212K-450KCUDA | Deep learning | Distributed Systems | GPU Performance | GPU Performance OptimizationEntry-level Full TimeSan Jose, California, United States17h ago
-
Artificial Intelligence | Data Modeling | Data Pipelines | Data Quality | Data Visualization401k match | Dental insurance | Life insurance | Medical insurance | Paid time offSenior-level Full TimeNew York17h ago
-
Backup and Restore | Blob Storage | Cluster communication | Cluster management | Crash diagnosticsSenior-level Full TimeSan Jose, California, United States18h ago
-
Data-Driven Decision Making | Data-driven | Decision Making | Deep learning | Distributed TrainingSenior-level Full TimeSunnyvale, CA19h ago
-
Production Engineer USD 178K-200KApache | Apache Spark | Application Programming | Application Programming Interfaces | C++Entry-level Full TimeMenlo Park, CA19h ago
-
Research Engineer - MSL FAIR Foundations USD 117K-173KBenchmarking | Code review | Data Pipelines | Distributed Systems | Language ModelEntry-level Full TimeMenlo Park, CA19h ago
-
AI Model Serving | AI model | Benchmarking | Cache Management | Data AnalysisSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA19h ago
-
Staff Software Engineer, AI Data Generation Platform USD 207K-300KComputer Vision | Data Engineering | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA19h ago
-
C plus plus | C++ | Cloud Spanner | Cloud Storage | Cloud platformSenior-level Full TimeSunnyvale, CA, USA19h ago