Software Engineer, Model Runtime
Tasks
- Build profiling observability and benchmarking tools
- Build scheduling and continuous batching
- Co design interfaces with kernel compiler and silicon teams
- Debug correctness performance and reliability issues
- Design LLM inference runtime
- Develop distributed execution strategies
- Enable new execution patterns and numerical formats
- Manage memory and KV cache
- Optimize latency throughput and utilization
- Orchestrate execution
Perks/Benefits
- N/A
Skills/Tech-stack
Benchmarking | C++ | Compilers | Continuous batching | Distributed Systems | KV cache | Kernel development | LLM Inference | Memory Management | Model Parallelism | Observability | Profiling | Python | Rust | Scheduling
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
AWS | Azure | CI/CD | Containers | DNS401k match | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSan Diego, CALIFORNIA, United States R21h ago
-
AI Engineer USD 180K-225KAWS | Agentic Workflows | Airflow | BigQuery | CI/CD401k match | Dental and vision coverage | Employee Assistance Program (EAP) | Employee discounts | HSA contributionsSenior-level Full TimeRemote, US R22h ago
-
Software Engineer, Data Plane Hybrid USD 147K-237KAgile | C# | C++ | Cloud Native | Cloud Native Network Function CNFSenior-level Full TimeWESTFORD, United States R1d ago
-
Principal Machine Learning Engineer USD 240K-420KAgent systems | Architecture | C plus plus | Compliance | Deep learning401k plan | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSanta Clara, California, United States R1d ago
-
Agent architecture | Benchmarking | Biomedicine | Context engineering | Data integrationFlexible working hours | Generous vacation | Hybrid work policy | Optional equity donation matching | Parental leaveMid-level Full TimeSan Francisco, CA | New York … R1d ago
-
Senior Data Engineer USD 140K-170KAPIs | Azure DevOps | DBT | Data Architecture | Data GovernanceDiscretionary bonus | Hybrid work schedule | Training and developmentSenior-level Full TimeNew York, New York, United States R1d ago
-
Staff Software Engineer, Data Warehouse Foundation USD 177K-364KApache Airflow | Apache Flink | Apache Spark | Apache Superset | Data PipelinesSenior-level Full TimePalo Alto, CA, US; Remote, US R1d ago
-
Senior Data Engineer USD 109K-164KAI Services | Azure AI | Azure AI Services | Azure Data | Azure Data Factory401k retirement plan | Employee assistance program | Health spending account | Life insurance | Long-term disabilitySenior-level Full TimeAustin, Texas, USA R1d ago
-
Senior Data Engineer- People Analytics USD 180K-207KAccess Control | Airflow | Amazon Redshift | CI/CD | Cloud platformSenior-level Full TimeUnited States R1d ago
-
Senior Data Visualization Engineer USD 114K-142KAdobe Illustrator | Adobe Photoshop | ArcGIS | Cartography | D3Employee assistance program | Home office reimbursement | Paid parental leave | Tuition reimbursement | Wellness programSenior-level Full TimeSan Francisco, CA R1d ago
-
Staff Software Engineer, Applied AI USD 200K-260KDataset curation | FastAPI | Golden datasets | Langchain | Langgraph401k retirement plan | Annual learning and development stipend | Employer HSA contribution | Equity incentive plan | Flexible vacation policySenior-level Full TimeU.S. Remote R1d ago
-
Principal Data Engineer USD 200K-280KAI Assisted Development | AWS | Airflow | Amazon Redshift | AzureSenior-level Full TimeDenver, CO or Long Beach, CA R1d ago
-
Data Engineer USD 152K-223KAWS | Apache Hadoop | Apache Kafka | Apache Spark | AzureHealth insurance | Opportunity for career growth | Paid Holidays | Paid time off | Training and developmentMid-level Full TimeUnited States R1d ago
-
Embedded Software Development Engineer (Senior Level) USD 120K-160KC# | C++ | Critical software | DO-178 | Embedded SystemsCritical illness insurance | Employee assistance program | Health insurance | Legal assistance | Mental health benefitsSenior-level Full TimeEast Hartford, CT, United States R1d ago
-
Embedded Software Verification Engineer (Mid level) USD 99K-134KC# | C++ | Case Development | Cause analysis | DO-178CBereavement leave | Critical illness insurance | Employee assistance program | Legal assistance | Medical, dental, and vision insuranceMid-level Full TimeEast Hartford, CT, United States R1d ago
-
C# | C++ | Case Development | Cause analysis | DO-178CCritical illness insurance | Dental insurance | Employee assistance program | Health insurance | Legal assistanceEntry-level Full TimeEast Hartford, CT, United States R1d ago
-
Principal Data Engineer USD 167K-225KAWS Glue | AWS Lambda | Amazon ECS | Amazon MWAA | Amazon RedshiftCompetitive health benefits | Educational reimbursements | Generous parental leave | Gym subsidies | Matched 401kSenior-level Full TimeRemote - US R1d ago
-
Principal Engineer, AI Architect USD 190K-210KCloud Architecture | Cloud Native | Conversational AI | Data analytics | Enterprise Architecture401k retirement plan | Development resources | Employee assistance program | Employee stock purchase program | Flexible paid time offSenior-level Full TimeNew York, New York R1d ago
-
Principal Engineer, AI Architect USD 190K-210KCloud Architecture | Conversational AI | Data analytics | Enterprise SaaS | GCP401k retirement plan | Employee assistance program | Employee stock purchase program | Flexible paid time off | Fully remoteSenior-level Full TimeNew York, New York R1d ago
-
Senior-level Full TimeRemote, Canada; Computer/Software; Canada, Canada; Remote, … R1d ago
-
Cloud Software Engineer 3 - (Cloud, Hadoop, Java) USD 228K-278KAnsible | Apache Kafka | Apache Lucene | Apache Solr | Avro401k matching | Educational assistance | Federal Holidays | Healthcare coverage | Paid time offSenior-level Full TimeFort Meade, Maryland R1d ago
-
Software Engineer, Data Platform Team USD 152K-179KAPI Development | Apache Airflow | Data Lineage | Data Modeling | Data Quality401k | Health insurance benefits | Paid time off | Quarterly in person surges | Remote-first work modelMid-level Full TimeRemote - USA R1d ago
-
Federal AI Solutions Engineer USD 85K-105KAI Risk Management Framework | AWS Bedrock | AWS CDK | AWS ECS | AWS EKS401k employer match | Career growth and mentorship opportunities | Dental insurance | Federal Holidays | Medical insuranceEntry-level Full TimeHybrid - McLean, VA, United States R1d ago
-
Principal Data Engineer - AI USD 139K-197KAmazon Kinesis | Amazon Redshift | Apache Airflow | Apache Flink | Apache IcebergHybrid work onsite two days per weekSenior-level Full TimePennsylvania-Remote, United States R1d ago
-
AI Engineer, Public Sector USD 157K-249KAgents | Embeddings | Faiss | FastAPI | Fine Tuning401k match | Flexible-hybrid work | Health insurance | Paid Company Holidays | Paid parental leaveMid-level Full TimeReston, Virginia, United States R1d ago