Software Engineer, Inference
SF Bay Area, CA, London, UK
USD 220K-300K (estimate) Entry-level Full Time
Tasks
- Architect end to end model serving deployment pipeline
- Automate test and maintain inference services
- Build and maintain CI CD pipelines for model checkpoints and platform components
- Build internal tooling to measure profile and track inference job lifetime
- Build robust queueing system for jobs with cluster availability and priority
- Build scheduling systems to leverage GPU resources and meet SLOs
- Build tracing system for end to end inference workload
- Collaborate across research engineering and infrastructure to optimize efficiency and deployments
- Create resilient artifact store for model checkpoints
- Enable hotswapping of models based on traffic patterns
- Integrate inference stack into online reinforcement learning pipeline
- Manage and optimize inference workloads across clusters and hardware providers
- Optimize deployment workflows to scale across machines
- Perform regression and precision testing across hardware platforms
- Ship model architectures by integrating into inference engine
Perks/Benefits
- N/A
Skills/Tech-stack
Amazon S3 | CD | CD pipelines | CI | CI/CD | CI/CD pipelines | CUDA | Docker | FFmpeg | Fleet Management | High Performance | High-Performance Computing | Huggingface | Infiniband | Kubernetes | Linux | NVLink | Performance Computing | PyTorch | Python | Queueing | RDMA | Redis | RoCE | SGLang | Scheduling | TensorRT-LLM | Traffic Control | VLLM
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Senior Software Engineer, Google Distributed Cloud AI USD 174K-252KBilling | Cloud Networking | Container Orchestration | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Software Engineer III, AI/ML, Foundational Lanes USD 147K-210KAutomation | C++ | Computational Geometry | Computer Vision | Data ProcessingSenior-level Full TimeMountain View, CA, USA3h ago
-
Agent systems | Agentic Frameworks | Backend Development | C plus plus | Data PipelinesSenior-level Full TimeSan Bruno, CA, USA; Mountain View, …3h ago
-
Senior Software Engineer, AI/ML, Google Workspace USD 174K-252KC plus plus | Code Reviews | Data Processing | Data Storage | DebuggingSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Agent systems | C++ | Database technologies | Distributed Systems | Embedding GenerationSenior-level Full TimeChicago, IL, USA; San Bruno, CA, …3h ago
-
Staff Software Engineer, Data Pipelines, DeepMind USD 207K-300KBenchmarking | C++ | Compilers | Data Structures | Data Structures and AlgorithmsBonus | Equity | Health insurance | Learning opportunities | Paid time offSenior-level Full TimeMountain View, CA, USA; New York, …3h ago
-
Software Engineer, On-Device Machine Learning USD 174K-252KAndroid | Compiler development | Core ML | Data Processing | DebuggingBonus | Equity | Health insurance | Paid time off | Retirement planMid-level Full TimeSunnyvale, CA, USA; New York, NY, …4h ago
-
Senior Data Engineer GBP 78K-100KAirflow | BigQuery | Billing | DBT | DagsterFree phone | Home office setup stipend | International data plan | Learning and development budget | Occasional travel for workshopsSenior-level Full TimeLondon, Berlin, Amsterdam7h ago
-
Embedded Systems Security Engineer USD 120K-150KARM TrustZone | ARMv7-A | AppArmor | Armv8-A | BareboxSenior-level Full TimeFoster City, California, United States11h ago
-
Databricks Platform Engineer USD 115K-135KAWS | Asset bundles | Audit Logs | Azure Databricks | CI/CD401k match | Employee assistance resources | Medical/Dental/Vision insurance | Paid time off | Professional development supportMid-level Full TimeRemote, United States R12h ago
-
Automation Scripting | Bare Metal | C plus plus | C# | CANHealth insurance | Professional development opportunities | Retirement plansSenior-level Full TimeHuntington Beach, CA14h ago
-
Analytics Engineer USD 140K-190KCI/CD | DBT | Data Modeling | Data Quality | Data Testing401k match | Flexible time off | Growth opportunities | Health benefits | Remote-friendlySenior-level Full TimeUnited States (Remote) R14h ago
-
AWS | Azure | Cloud platform | Data Analysis | Data ModelingCycle to work scheme | Employee assistance programme | Eye care | Free breakfast | Gym discountEntry-level Full TimeHavas Village London - 35, United … R15h ago
-
IT Storage Engineer USD 135K-175KAnsible | DFS | Enterprise networking | FlashArray | FlashBlade401k retirement plan | Company shuttles | Employee discounts | Employee stock purchase plan | Life insuranceMid-level Full TimeRedmond, WA16h ago
-
IT Storage Engineer USD 135K-175KAnsible | CIFS | DFS | FlashArray | FlashBlade401k retirement plan | Life insurance | Long-term disability | Medical, dental & vision coverage | Paid HolidaysMid-level Full TimeHawthorne, CA16h ago
-
IT Storage Engineer USD 130K-175KAlerting | Ansible | CIFS | Capacity Planning | Configuration ManagementExtended hours and weekends | On-call rotationMid-level Full TimeStarbase, TX16h ago
-
IT Storage Engineer USD 130K-175KAnsible | CIFS | DFS | FlashArray | FlashBladeExtended hours | On-call rotation | Physical hardware installation | Weekend workMid-level Full TimeCape Canaveral, FL16h ago
-
IT Storage Engineer USD 130K-175KAlerting | Ansible | CIFS | Capacity Planning | Configuration ManagementExtended hours and weekend availability | Occasional physical hardware installation | On-call rotationMid-level Full TimeBastrop, TX16h ago
-
Full Stack Software Engineer, AI & Data Platforms, IS&T USD 143K-250KAWS | Azure | CSS | GCP | GRPCMid-level Full TimeAustin16h ago
-
Software Engineer, Data Integration USD 165K-225KAWS | Automated testing | Azure | Cloud Computing | Data IngestionSenior-level Full TimeRemote - US Only R16h ago
-
AI Engineer USD 85K-128KAlerting | Azure | CI/CD | Distributed Systems | Docker401k | Adoption Assistance | Dental insurance | Disability insurance | Employer Paid Life AD D DisabilitySenior-level Full TimeMN, United States16h ago
-
Software Engineer - AI Developer Productivity USD 165K-330KAudit Logging | Authentication | Authorization | CI/CD | Context ManagementCompany facilitated 401k | Fertility and family building stipend | Flexible PTO | Medical dental vision insurance 100 percent | Paid parental leaveSenior-level Full TimeSan Francisco R17h ago
-
Telemetry Data Engineer USD 110K-128KAnomaly Detection | Azure Data | Azure Data Explorer | Data Engineering | Data Explorer401k retirement plan | Medical/Dental/VisionMid-level ContractRedmond, Washington, USA R17h ago
-
Software Engineer, Applied AI Infrastructure USD 193K-352KAttention | Batching | CI Failure Attribution | Cloud infrastructure | Context handlingSenior-level Full TimeMountain View, California (HQ)17h ago
-
Senior AI Engineer USD 176K-258KAgentic Workflows | Agile | Cloud Architecture | Code review | DatabricksCell phone allowance | Comprehensive health coverage | Equity grants | Growth-focused environment | Home office setup allowanceSenior-level Full TimeRemote - US R17h ago