Inference Engineer
Tasks
- Build and operate production model serving systems
- Design architecture decisions and engineering standards
- Improve scalability and operational maturity
- Investigate resolve performance reliability and capacity issues
- Maximize GPU utilization with predictable performance
- Monitor alert and maintain reliable inference services
- Optimize inference infrastructure for throughput latency scalability cost
Perks/Benefits
Skills/Tech-stack
API Development | Alerting | Batching | Caching | Cloud infrastructure | Distributed Systems | GPU Computing | GPU scheduling | High Throughput | Inference Server | Kubernetes | Latency optimization | Low Latency | Machine Learning | Model Serving | Monitoring | NVIDIA Triton | NVIDIA Triton Inference | NVIDIA Triton Inference Server | Performance Tuning | Quantization | Scalability | TensorRT-LLM | Token Throughput | Triton Inference Server | VLLM
Education
N/A
Related jobs
-
Allocation Optimization | Auditability | Bin packing | CPLEX | Constraint ProgrammingSenior-level Full TimeMenlo Park, CA2h ago
-
API Integration | Access Control | Automated Evaluation | BigQuery | CassandraEquity compensation | Health insurance | Paid time off | Retirement planSenior-level Full TimeSunnyvale, CA, USA2h ago
-
Senior Software Engineer, AI/ML, Google Cloud Platforms USD 174K-252KC++ | Data Processing | Language Processing | ML Infrastructure | Machine LearningSenior-level Full TimeNew York, NY, USA; Sunnyvale, CA, …2h ago
-
Senior Data Application Engineer, Core USD 163K-236KCloud Data | Cloud data lakehouse | Data Contracts | Data Governance | Data LakehouseSenior-level Full TimeSan Jose, CA, USA2h ago
-
Software Engineer, GDC AI Applications and Agents USD 147K-210KAPIs | Cloud Computing | Data Structures | Data Structures and Algorithms | Distributed SystemsBenefits | Bonus target | EquityMid-level Full TimeSunnyvale, CA, USA2h ago
-
API Design | Autoscaling | Data analytics | Databases | Distributed ComputingSenior-level Full TimeSunnyvale, CA, USA2h ago
-
Data Structures | Data Structures and Algorithms | Embedding | Experimentation | Language ProcessingSenior-level Full TimeSan Bruno, CA, USA3h ago
-
Cloud infrastructure | Distributed Systems | GPU | JAX | Machine LearningMid-level Full TimeUnited States R8h ago
-
LLM Application Engineer USD 130K-175KAPIs | Agent Frameworks | Databases | Distributed Systems | EvaluationMid-level Full TimeUnited States R8h ago
-
Staff Software Engineer AI/ML USD 163K-253KA/B | A/B Testing | Autogen | B testing | Batched Inference401k | Adoption support | Charitable giving match | Fertility care stipend | Flexible work environmentSenior-level Full TimeSan Jose, California, United States9h ago
-
Senior/Staff - Agent Platform (Cortex Code) USD 236K-309KApache Airflow | DBT | Data Analysis | Data Engineering | Data ModelingSenior-level Full TimeUS-CA-Menlo Park10h ago
-
Machine Leaning Performance Engineer (Inference) USD 200K-300KC++ | CUBLAS | CUDA | CUDA C | CutlassFitness Events | Free meals | Hybrid work options | Paid time off | Volunteer opportunitiesMid-level Full TimeNew York R11h ago
-
Machine Learning Research Engineer USD 200K-300KAI Agents | API Design | Automation | Dask | Data pipelineHybrid work | Paid time off | Paid wellness expenses | Volunteer opportunities | Workshops and continuous learningMid-level Full TimeNew York R11h ago
-
Agent Orchestration | Agentic AI | Apache Airflow | Argo Workflows | Artifact versioning401k employer match | Comprehensive healthcare benefits | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R13h ago
-
Staff, Advanced Analytics, Incubations USD 180K-221KA/B | A/B Testing | Attribution | Attribution Modeling | B testingSenior-level Full TimeUnited States R13h ago
-
Senior Software Engineer (Data & AI Solutions) USD 125K-156KAWS | Apache Airflow | Apache Spark | BAM | Bioinformatics401k benefits | Commuter benefits | Employee referral program | Fertility care benefits | Free testingSenior-level Full TimeUS Remote R14h ago
-
AI | Apache Cassandra | Caching | Capacity Planning | Data ModelingField testing opportunities | Hybrid work model | Offsite collaborationMid-level Full TimeSan Francisco, CA; Los Angeles, CA; …14h ago
-
C# | C++ | Embedded Systems | Go | JavaSenior-level Full TimeCupertino15h ago
-
Staff Software Engineer, Backend (Online Storage) USD 230K-290KAWS | Capacity Planning | Database Benchmarking | Disaster Recovery | Distributed SQLDental and vision coverage | Employee stock purchase plan | Flexible spending wallets | Health care coverage | Remote work flexibilitySenior-level Full TimeRemote US R15h ago
-
Audio Processing | C plus plus | C# | Computer Vision | Image ProcessingMid-level Full TimeCupertino15h ago
-
Staff Machine Learning Engineer - Moloco Commerce Media USD 172K-244KA/B | A/B Testing | B testing | CTR Prediction | Conversion Rate401k company match | Flexible time off | Life insurance | Long-term disability | Medical/Dental/Vision insuranceSenior-level Full TimeMenlo Park, California, United States16h ago
-
C# | C++ | CI/CD | CUDA | Computer Architecture401k | Dental insurance | FSA | Health insurance | Office snacksSenior-level Full TimeMountain View, California, United States16h ago
-
Data Engineer - Databricks USD 172K-237KACID transactions | Airflow | Auto Loader | Azure Data | Azure Data FactoryMid-level Full TimeChicago, IL, United States16h ago
-
Data Engineer USD 170K-209KAWS | Amazon Kinesis | CI/CD | Compliance Frameworks | Container OrchestrationSenior-level Full TimeUS - Remote; Canada - Remote R16h ago
-
Senior AI Systems Engineer – LLM & Agentic Systems USD 140K-160KAWS | Agentic AI | Alerting | Drift Detection | EmbeddingsDaily group classes | Dry cleaning benefits | Free onsite fitness center | Free shuttle transportation | Gourmet cafeteriaSenior-level Full TimeEnglewood Cliffs, NEW JERSEY, United States R17h ago