Software Engineer, Inference
SF Bay Area, CA, London, UK
USD 220K-300K (estimate) Entry-level Full Time
Tasks
- Architect end to end model serving deployment pipeline
- Automate test and maintain inference services
- Build and maintain CI CD pipelines for model checkpoints and platform components
- Build internal tooling to measure profile and track inference job lifetime
- Build robust queueing system for jobs with cluster availability and priority
- Build scheduling systems to leverage GPU resources and meet SLOs
- Build tracing system for end to end inference workload
- Collaborate across research engineering and infrastructure to optimize efficiency and deployments
- Create resilient artifact store for model checkpoints
- Enable hotswapping of models based on traffic patterns
- Integrate inference stack into online reinforcement learning pipeline
- Manage and optimize inference workloads across clusters and hardware providers
- Optimize deployment workflows to scale across machines
- Perform regression and precision testing across hardware platforms
- Ship model architectures by integrating into inference engine
Perks/Benefits
- N/A
Skills/Tech-stack
Amazon S3 | CD | CD pipelines | CI | CI/CD | CI/CD pipelines | CUDA | Docker | FFmpeg | Fleet Management | High Performance | High-Performance Computing | Huggingface | Infiniband | Kubernetes | Linux | NVLink | Performance Computing | PyTorch | Python | Queueing | RDMA | Redis | RoCE | SGLang | Scheduling | TensorRT-LLM | Traffic Control | VLLM
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Senior Databricks Platform Engineer USD 140K-165KAWS | Active Directory | Apache Hive | Apache Spark | Audit LoggingSenior-level Full TimeArlington, VA, United States3h ago
-
Senior Data Engineer-JT0224 USD 115K-175K.Net Core | .Net Framework | AWS CloudFormation | Apache Airflow | Azure401k match | Career growth opportunities | Dental insurance | Employee resource groups | Health insuranceSenior-level Full TimeRemote, United States R3h ago
-
Senior Machine Learning Engineer USD 140K-224KAIOps | AWS | Apache Spark | Cloud platform | Data Analysis401k match | Dental insurance | Flexible paid time off | Life insurance | Medical insuranceSenior-level Full TimeRemote, United States R3h ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL5h ago
-
Senior-level ContractRaleigh, United States5h ago
-
SYSTEM ENGINEER - Computer Network Support - AI/ML - 6+ yrs of Experience - TS/SCI w/Poly clearance is required - ES - 032726-2 A USD 136K-140KAI/ML | Agile | Confluence | Jira | LLM401k retirement plan | Dental insurance | Disability insurance | Federal Holidays | Floating holidaysMid-level Full TimeFort George G Meade, United States6h ago
-
AI Platform Engineer USD 108K-215KAutogen | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Key VaultMid-level Full TimeBelmont, NC, United States6h ago
-
AI Platform Engineer USD 108K-215KAI Agents | Agent Frameworks | Autogen | Azure Data | Azure Data LakeMid-level Full TimeBelmont, NC, United States6h ago
-
AI experimentation | Alerting | Artificial Intelligence | Data Annotation | Data ArchitectureSenior-level Full TimeNew York6h ago
-
Senior Software Engineer (C++/DS/Algorithm) USD 119K-164KAlgorithms | C# | C++ | Compiler concepts | Compiler optimizationsHealth and wellness benefits | Hybrid work modeSenior-level Full TimeAustin - Texas - United States …6h ago
-
Asynchronous training | Containerization | Curriculum learning | Distributed Systems | Distributed TrainingSenior-level Full TimeSF Bay Area, CA, Remote, US, … R7h ago
-
Data Engineer GBP 36K-45KAWS | AWS Glue | Airflow | Azure | DBTAnnual paid time off | Discounted flights | Flexible benefits | Holiday vouchers | Hybrid workMid-level Full TimeUnited Kingdom-London-London Luton Airport7h ago
-
Software Engineer - TikTok AI Search Infrastructure USD 212K-389KAlgorithm Design | C++ | DAG | Data Architecture | Data ProcessingSenior-level Full TimeSan Jose, California, United States8h ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 78K-176KAPIs | Agile methodologies | Backend Development | Cloud Computing | Data Preparation401k | Employee discount program | Employee referral rewards | Flexible spending account | Flexible work scheduleMid-level Full TimeFort Belvoir, VA, US R8h ago
-
Software Engineer, Machine Learning GBP 100K-130KData Structures | Data structures algorithms | Graph theory | Information Retrieval | Language ProcessingSenior-level Full TimeLondon, UK9h ago
-
Software Engineer, Infrastructure GBP 180K-225KAI | Agent Orchestration | Bias Mitigation | C# | C++Senior-level Full TimeZurich, Switzerland | Tel Aviv, Israel …9h ago
-
Senior-level Full TimeLondon, UK9h ago
-
Software Engineer, Systems ML Engineering USD 170K-251KAlerting | Benchmarking | C++ | CUDA | DashboardingSenior-level Full TimeSunnyvale, CA | Bellevue, WA | …9h ago
-
AI/HPC Network Performance Engineer USD 147K-226KAI Training | Alerting | Auto-remediation | C++ | Configuration ManagementOncall rotationSenior-level Full TimeMenlo Park, CA9h ago
-
Senior-level Full TimeLondon, UK9h ago
-
Forward Deployed Engineer III, Google Cloud, Applied AI USD 174K-253KAPI Integration | Agent systems | Agentic Workflows | Chatbots | Cloud platformBenefits | Bonus | Equity | Travel up to 50 percentSenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …9h ago
-
Research Software Engineer, Multimodal AI USD 174K-253KAgent Orchestration | Audio | C++ | Few-Shot Learning | Few-shotMid-level Full TimeSan Jose, CA, USA9h ago
-
ML Engineer, GenAI Ads, Search Personalization USD 207K-301KC++ | Clustering Algorithms | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA9h ago
-
Data Filtering | Data Processing | Data debugging | Deep learning | Distributed ComputingSenior-level Full TimeLondon, UK9h ago