Software Engineer, Inference Runtime
Tasks
- Benchmark and diagnose correctness and performance issues
- Bring up model architectures and multimodal models
- Build model loading batching scheduling caching and distributed execution
- Contribute upstream to open source inference projects
- Integrate inference engines and runtime capabilities
- Maintain inference stack on device and in cloud
- Optimize model execution for CPU and GPU targets
Perks/Benefits
Skills/Tech-stack
C++ | CPU Profiling | CUDA | Executorch | GPU Profiling | Llama.cpp | MLX | Machine Learning | Metal | Model Inference | PyTorch | Python | ROCm | SGLang | TensorRT-LLM | Transformer Architectures | VLLM | Vulkan
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R29d ago
-
Analytics Engineer USD 130K-170KCI/CD | DBT | Data Modeling | Data Quality | Data Quality Testing401k match | Flexible time off | Health benefits day one | Paid caregiver leave | Paid parental leaveSenior-level Full TimeUnited States (Remote) R9h ago
-
Staff Machine Learning Engineer, Shopping Ads USD 230K-322KConversion modeling | Data labeling | Deep learning | Distributed Systems | Feature Engineering401k employer match | Comprehensive healthcare benefits | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - United States R9h ago
-
Data Engineer USD 81K-105KAPIs | AWS | Azure | Cloud Data | Cloud Data Platforms401k match | Full-flex work week | Health and wellness benefits | Hybrid work schedule | Life insuranceMid-level Full TimeUSA VA Falls Church - 3150 … R10h ago
-
Staff ML Engineer, Perception Research USD 251K-310K3D Perception | Autoregressive models | Data Pipelines | Data parallelism | Diffusion Models401k match | Baby bonding leave | Dental insurance | Disability insurance | Health insuranceSenior-level Full TimeMountain View, CA, USA ; New … R10h ago
-
Senior ML Engineer, Perception Research USD 213K-263K3D Perception | Autoregressive models | Camera | Computer Vision | Data Pipelines401k matching | Baby bonding leave | Dental insurance | Disability insurance | Health insuranceSenior-level Full TimeMountain View, CA, USA ; New … R10h ago
-
Predictive Analytics Consultant USD 69K-101KAWS IAM | AWS Lambda | AWS SageMaker | AWS Step Functions | Amazon CloudWatchSenior-level Full TimeUS Remote R12h ago
-
AI Engineer I USD 90K-119KAPI Integration | Azure | CI/CD | Chroma | Docker401k match | Dental insurance | Holidays | Medical insurance | Personal Time OffEntry-level Full TimeRemote, CA, USA R12h ago
-
Machine Learning Engineer - New Grad 2026 USD 150K-175KDeep learning | Experimentation | Java | Knowledge graph | Machine LearningCommuter benefits | Dedicated volunteer days | Employee resource groups | Flexible paid time off | Gender-affirming careEntry-level Full TimeDallas, TX; New York, NY; San … R15h ago
-
Data Engineer III/Senior (Analytics) USD 130K-170KAnalytics engineering | Azure | CCPA | CI/CD | DORA Metrics401k retirement plan | Dental insurance | Employee stock purchase plan | Health insurance | Life insuranceSenior-level Full TimeIrving, TX, United States R15h ago
-
AI Search | AI Security | AWS | AWS Bedrock | Agent memoryHybrid work | In-person interviewSenior-level Contract Full TimeJersey City, NJ, United States R16h ago
-
Lead AI Software Engineer, AI Platform & Architecture USD 180K-225KBackend Development | CI/CD | Cloud Architecture | Cost Optimization | Distributed SystemsDental insurance | Flexible time off | Medical insurance | Paid parental leave | Sabbatical after 5 yearsSenior-level Full TimeUnited States (Remote-First) R17h ago
-
Senior-level Full TimeNew York Office R17h ago
-
Principal AI Engineer – Agentic AI / Generative AI | Irvine, CA (Hybrid) | Contract to Hire | AS A USD 100K-150KAI Agents | API | AWS | Agent systems | CI/CDContract-to-hire | Hybrid workSenior-level Full TimeIrvine, CA R18h ago
-
Staff Engineer, Query Optimization USD 151K-297KAlgorithms | Backend Engineering | C++ | Code Quality | Compilers401k plan | Employee stock purchase program | Fertility and adoption assistance | Flexible paid time off | Health benefitsSenior-level Full TimeUnited States R19h ago
-
Calibration | Data Generation | Deep learning | Distillation | EHREquity | Health benefits | High autonomySenior-level Full TimeNew York, NY, United States R19h ago
-
Senior Data Engineer ID75059 USD 155K-170KAPI Integration | AWS | Apache Airflow | Apache Spark | AvroEducation budget | Exciting projects | Fitness budget | Flexible schedule | MentorshipSenior-level Full TimeBaltimore, United States R20h ago
-
Senior Data Engineer ID75059 USD 155K-170KAPI Integration | AWS | Apache Airflow | Apache Spark | AvroEducation budget | Exciting projects | Fitness budget | Flexible schedule | MentorshipSenior-level Full TimeHouston, United States R20h ago
-
Senior Data Engineer ID75059 USD 155K-170KAPI Integration | AWS | Airflow | Apache Kafka | AvroFlexible schedule | Mentorship | Personalized growth roadmaps | Professional growth | Remote and office optionsSenior-level Full TimeWest Palm Beach, United States R20h ago
-
Senior Data Engineer ID75059 USD 155K-170KAWS | Apache Airflow | Apache Spark | Avro | DuckDBEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeBoca Raton, United States R20h ago
-
Senior Data Engineer ID75059 USD 155K-170KAWS | Airflow | Apache Kafka | Apache Spark | AvroFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeJersey City, United States R20h ago
-
Senior Data Engineer ID75059 USD 155K-170KAPI Integration | AWS | AWS Glue | Apache Airflow | Apache SparkEducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsSenior-level Full TimeTallahassee, United States R20h ago
-
Senior Data Engineer ID75059 USD 155K-170KAPI Integration | AWS | Airflow | Apache Spark | AvroFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeJacksonville, United States R20h ago
-
Senior Data Engineer ID75059 USD 155K-170KAWS | Airflow | Amazon SageMaker | Amazon SageMaker Studio | Apache SparkEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeLos Angeles, United States R20h ago
-
Senior Data Engineer ID75059 USD 155K-170KAPIs | AWS | Airflow | Avro | DuckDBEducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapSenior-level Full TimeFort Lauderdale, United States R20h ago