Staff Software Engineer, AI Runtime
Mountain View, California; San Francisco, California
USD 190K-265K Senior-level Full Time
Tasks
- Build checkpoint restore and fault tolerance for long running jobs
- Define APIs CLI and developer experience for training jobs
- Deliver resilience observability for multi node jobs
- Drive architecture and evolution of managed GPU training platform
- Implement GPU scheduling dynamic routing and high throughput data loading
- Lead end to end engineering from design to production rollout
- Mentor engineers and lead technical discussions
- Solve multi node orchestration and distributed training performance problems
Perks/Benefits
- N/A
Skills/Tech-stack
Algorithms | Automatic Recovery | Checkpointing | Collective communication | Data Structures | Data loading | Data parallelism | DeepSpeed | Distributed Systems | FSDP | Failure Detection | GPU Computing | High Performance | High Throughput | High Throughput Data Loading | High-Performance Computing | High-throughput data | Infiniband | Machine Learning | Megatron | Multi-tenant | Multi-tenant platform | NVLink | Orchestration | Performance Computing | Pipeline parallelism | PyTorch | RoCE | SLAs | SLOs | Sequence parallelism | System design | Tensor Parallelism
Education
Roles
Regions
Countries
States
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R17d ago
-
AI | Adversarial Machine Learning | Bot Detection | Data Analysis | Data IntegrityMid-level Full TimeMountain View, CA, USA9h ago
-
Staff Software Engineer, AI/ML, Google Cloud Security USD 207K-301KAgentic AI | Cloud platform | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeReston, VA, USA; New York, NY, …9h ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R15h ago
-
Senior Principal AI Engineer USD 152K-254KData Processing | Git | Hugging Face | Language Models | Language ProcessingAdoption Assistance | Dental vision prescription coverage | Disability benefits | Employee assistance program | Health benefitsSenior-level Full TimeCambridge, United States20h ago
-
Senior Machine Learning Engineer USD 134K-168KData Drift | Databricks | Ethical AI | Feature Engineering | KubernetesSenior-level Full TimeNew York, NY, United States20h ago
-
Storage and Data Protection SME USD 105K-158KActive Directory | Bandwidth Optimization | Brocade | CTERA | Cisco UCSDental insurance | Health insurance | Life insurance | Paid time off | Retirement planMid-level Full TimeJoint Base Pearl Harbor-Hickam, Hawaii, United …20h ago
-
Senior Software Engineer - Database USD 165K-225KC# | C++ | Databases | Distributed Systems | Fault ToleranceSenior-level Full TimeNew York, NY, US1d ago
-
Senior Software Engineer - Database USD 165K-225KAlgorithms | C# | C++ | Concurrency | Data StructuresSenior-level Full TimeNew York, NY, US1d ago
-
AWS | AWS CloudFormation | Amazon ECS | Automation | CI/CD401-k plan | Company match | Employee stock purchase plan | Healthcare benefits | Life insuranceMid-level Full TimeAtlanta, GA, United States1d ago
-
Senior Staff Generative AI Scientist USD 185K-210KAWS | Apache Spark | Artificial Intelligence | Azure | Cloud platform401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeRemote, United States R1d ago
-
Machine Learning Engineer USD 117K-177KAWS | Azure | Cloud Computing | Data Visualization | Google CloudMid-level Full TimeSt. Louis, Missouri, United States1d ago
-
Software Data Engineer 4 (Enterprise Infrastructure) USD 150K-237KCloud Security | Cloud infrastructure | IT Operations | Linux | Network Automation401k matching | Career counseling | Career development | Dental insurance | Health insuranceSenior-level Full TimeLaurel, MD1d ago
-
Software Data Engineer 3 (Enterprise Infrastructure) USD 140K-220KRequirements Analysis | Software Design | Software Engineering | System Performance | System designCareer development | Dental insurance | Employee referral bonuses | Health insurance | Paid HolidaysMid-level Full TimeLaurel, MD1d ago
-
Agile Development | Artificial Intelligence | Confluence | Jira | LLMCareer development | Education & training | Health, dental & vision plans | Holidays | Paid time offSenior-level Full TimeFort Meade, MD1d ago
-
Software Engineer - X Data USD 125K-400KApache Flink | Apache Kafka | Apache Spark | BigQuery | Cause analysis401k retirement plan | Dental insurance | Discounts | Health insurance | Life insuranceMid-level Full TimePalo Alto, CA1d ago
-
Site Reliability Engineer, Inference Infrastructure USD 165K-267KAmazon Web Services | C plus plus | Cloud platform | Distributed Systems | GPU401k matching | Co working Benefit | Company offsite | Education and learning stipend | Health and dental benefitsMid-level Full TimeToronto1d ago
-
Staff Software Engineer, Inference Infrastructure USD 160K-220KAmazon Web Services | Azure | C++ | Cloud platform | Distributed Systems401k | Annual company offsite | Arts and culture budget | Education and learning stipend | Health and dental benefitsSenior-level Full TimeSan Francisco1d ago
-
AI Engineer – Routing & Network Optimization USD 80K-210KA Star | A Star Algorithm | AMPL | Bellman Ford Algorithm | CPLEX401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
AI Software Engineer USD 80K-210KAWS Kinesis | Apache Airflow | Apache Kafka | C# | C++401k | Equity grant | Full healthcare coverage | Unlimited PTOSenior-level Full TimeEl Segundo, CA1d ago
-
Applied AI Engineer USD 204K-352KBackend Development | Cloud infrastructure | Deep learning | Docker | Fine TuningAnnual membership | Centralized coffee stipend | Flexible PTO | Health and wellness stipend | Health, dental, and vision insuranceSenior-level Full TimeNew York, NY (HQ)1d ago
-
Senior Applied Scientist - Search USD 200K-200KData Science | Embedding | Fine Tuning | Hybrid search | Information Retrieval401k retirement | Annual leave | Equity package | Health coverage | Hybrid scheduleSenior-level Full TimeNew York City R1d ago
-
ML Research Engineer - Hardware Codesign USD 185K-455KC++ | CUDA | Floating point | Floating point numerics | Functional simulationHybrid work schedule | Relocation assistanceSenior-level Full TimeSan Francisco1d ago
-
Senior Quantum Applications Engineer - QEC USD 120K-258KCUDA-Q | Decoder algorithms | Docker | End to End | End-to-End TestingSenior-level Full TimeNew Haven, CT1d ago
-
Staff AI engineer USD 170K-250KAI Evaluation | AWS | Agent Orchestration | Caching | Distributed SystemsFlexible working hours | Hybrid work culture | Unlimited time offSenior-level Full TimeSan Francisco1d ago