Software Engineer - GenAI inference
Tasks
- Build instrumentation profiling and tracing tools
- Design and implement inference engine
- Document learnings and share best practices
- Ensure reliability reproducibility and fault tolerance
- Handle communication overhead for inference workloads
- Implement routing batching scheduling memory management and dynamic loading
- Integrate new model architectures into engine
- Optimize latency throughput memory efficiency and hardware utilization
- Optimize model serving stack for large scale LLM inference
- Orchestrate distributed inference across nodes and balance load
Perks/Benefits
- N/A
Skills/Tech-stack
Attention Mechanism | CUDA | CUDA cuBLAS | CUDA/cuDNN | GPU Programming | Instrumentation | ML inference | Memory partitioning | Mixture of Experts | NCCL | Profiling | Quantization | Queuing | RPC | Sharding | Sparse operations | Tracing
Education
Roles
Regions
Countries
States
Related jobs
-
.NET | AI | AI SSDLC | Agentic Workloads | Argo CDMid-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …17h ago
-
Agentic Workflows | Authentication | Authorization | Circuit Breaker | Embedding401k match | Employee assistance program | Health insurance | Paid time off | Paid volunteer daysExecutive-level Full TimePrinceton, New Jersey, United States1d ago
-
Applied AI ML - Senior Associate USD 177K-215KAccess Controls | Analytics | Audit Logging | CI/CD | DockerBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeNY, United States1d ago
-
Staff Embedded Software Engineer USD 160K-264KBash | C++ | CPU GPU | CPU GPU Performance | CPU GPU Performance ProfilingSenior-level Full TimeSunnyvale, CA, United States1d ago
-
Senior-level Full TimeUS Remote R1d ago
-
Applied AI Site Reliability Engineer III - PxE Talent USD 102K-210K.NET | AI Operations | ArgoCD | Automated testing | BashSenior-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …1d ago
-
Lead Applied AI Site Reliability Engineer II - PxE ERM USD 113K-232K.NET | ArgoCD | Bash | C# | CI/CDMid-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …1d ago
-
Forward Deployed Engineer IV, GenAI, Public Sector USD 207K-300KAI Pipelines | API Integration | Agent systems | BigQuery | CI/CDSenior-level Full TimeReston, VA, USA1d ago
-
Research Engineer, Knowledge Graph Intelligence USD 175K-250KAttention Mechanisms | Data Augmentation | Data Preprocessing | Deep learning | Embeddings401k match | Employee affinity groups | Family leave | Health care benefits | Parental leaveMid-level Full TimeNew York, NY2d ago
-
Software Engineer II - Embedded Software (Onsite) USD 68K-131KAgile | AppArmor | Artifactory | C# | C++Relocation assistanceMid-level Full TimeUS-TX-MCKINNEY-513WD ~ 2501 W University Dr …2d ago
-
C++ | CUDA | EO/IR | Earth Observation | Embedded SystemsSenior-level Full TimeFort Wayne, IN2d ago
-
AI and ML Engineer, Mid USD 77K-176KAWS | Anomaly Detection | CI/CD | CUDA | Computer VisionDependent care | Paid leave | Professional development | Tuition assistance | Work-life programsMid-level Full TimeUSA, OH, Wright Patterson AFB (4180 …2d ago
-
Senior AI/ML Engineer - Agentic AI USD 131K-271KAPI Design | AWQ | Agent systems | Arize | Asynchronous programmingSenior-level Full TimeNewport Beach, CA, US, 926602d ago
-
Forward Deployed AI Solutions Engineer USD 105K-132KAPIs | Auditability | Authentication | Command Line | Command-line Interface401k benefits | Baby bonding leave | Commuter benefits | Disability coverage | Employee referral programMid-level Full TimeUS Remote R2d ago
-
Staff Forward Deployed AI Solutions Engineer USD 152K-190KAPIs | Auditability | Auth | CLI | Cloud platform401k | Commuter benefits | Dental insurance | Disability insurance | Fertility care benefitsSenior-level Full TimeUS Remote R2d ago
-
Staff Machine Learning Engineer - League of Legends USD 178K-256KCausal Inference | Contextual Bandits | Decision logic | Experimentation | Feature Stores401k match | Dental insurance | Flexible work schedule | Life insurance | Medical insuranceSenior-level Full TimeLos Angeles, USA2d ago
-
Applied AI ML Lead - Agent Builder Platform USD 177K-215KAccess Control | Analytics | Audit Logging | Code review | Continuous DeliveryBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimeJersey City, NJ, United States2d ago
-
Software Engineer, Power Optimization (Starlink) USD 125K-165KAlgorithms | Build Automation | C# | C++ | Continuous Deployment401k retirement plan | Company shuttles | Employee stock purchase plan | Life insurance | Long-term disability insuranceMid-level Full TimeRedmond, WA2d ago
-
Member of Technical Staff - Data Platform USD 180K-440KAPI Development | Apache Flink | Apache Kafka | Apache Spark | Data Pipelines401k retirement plan | Dental insurance | Discounts | Equity | Health insuranceSenior-level Full TimePalo Alto, CA2d ago
-
Staff/Principal DevOps Engineer, AI Inference USD 192K-272KAWS IAM | AWS S3 | Amazon EC2 | Amazon EKS | Amazon VPCCommuter benefits | Company Subsidized Lunch Program | Company holidays | Dental insurance | Educational assistanceSenior-level Full TimeCambridge, MA USA2d ago
-
Autoregressive models | Computer Vision | Diffusion Models | Efficient Inference | Generative AISenior-level Full TimeSeattle, Washington, United States2d ago
-
CUDA | Distributed Systems | GPU | Language Models | Large Language ModelsCareer growth | Global team collaborationEntry-level Full TimeSan Jose, California, United States2d ago
-
AI and DB | Automation | Backup and Restore | CPU Optimization | CXLMid-level Full TimeSeattle, Washington, United States2d ago
-
Causal Inference | Cross-modal fusion | DPO | Data Modeling | Deep reinforcement learningEntry-level Full TimeSeattle, Washington, United States2d ago
-
Large Model Training Acceleration Engineer USD 156K-316KData parallelism | Deep learning | Distributed Training | Model Optimization | Model ParallelismSenior-level Full TimeSan Jose, California, United States2d ago