Senior Inference Engineer
Tasks
- Apply quantization batching caching and request routing
- Build and deploy production grade LLM inference systems
- Collaborate on operational practices for production GPU infrastructure
- Collaborate with Product to translate requirements into inference capabilities
- Define and execute inference infrastructure technical roadmap
- Design implement and operate model serving infrastructure
- Develop scalable Python or Golang production infrastructure
- Establish and own inference platform evolution
- Evaluate emerging inference technologies
- Optimize inference workloads for latency throughput reliability and cost
Perks/Benefits
- Collaborate with product and technical leadership
- Flexible working schedule
- Health, dental, vision, and life insurance
- Ownership over architecture and roadmap
- Remote work flexibility
- Remote-first environment
Skills/Tech-stack
Batching | CUDA | Caching | Docker | Fine Tuning | GPU Software Stack | GPU software | Golang | Kubernetes | LLM serving | Model Architecture | NCCL | PyTorch | Python | Quantization | Request Routing | SGLang | Software stack | TensorRT-LLM | Transformers | VLLM
Education
N/A
Related jobs
-
AWS | Amazon Kinesis | Apache Airflow | Apache Spark | Cloud platformFull-time opportunity | Fully remote workMid-level Full TimeCanada R1d ago
-
Senior Analyst, Quantitative Data Science CAD 70K-110KAirflow | Analytics engineering | BigQuery | CI/CD | Cloud StorageSenior-level Full TimeQuebec / 1080, Grande Allee West … R1d ago
-
Senior Applied AI Engineer – GenAI Systems CAD 129K-179KAccess Control | Backtesting | CI/CD | Data Lineage | Data QualityHybrid work environment | Inclusion and Diversity support | Learning and development opportunities | Well-being supportSenior-level Full TimeCAN, Ontario, Toronto, 200 Bloor Street … R1d ago
-
Machine Learning Engineer CAD 94K-144KAIOps | Alerting | Azure DevOps | Azure DevOps Boards | Azure Key VaultDental insurance | Disability insurance | Employee assistance program | Health insurance | Life insuranceMid-level Full TimeCAN, Ontario, Toronto, 250 Bloor Street … R1d ago
-
Staff Data Engineer CAD 120K-144KApache Kafka | CI/CD | Data Modeling | Data Observability | Data QualityDiversity and inclusion working groups | Flexible working practices | Paid volunteer time off | Parental leaveSenior-level Full TimeCanada - Remote R1d ago
-
Staff + Senior Software Engineer, Inference CAD 163K-200KAWS | Autoscaling | Azure | Batching | CachingEquity donation matching | Flexible working hours | Generous vacation | Parental leaveSenior-level Full TimeOntario, CAN R1d ago
-
Staff Software Engineer, Data (AI) USD 210K-260KAI Evaluation | API | AWS | Amazon Redshift | Data EnrichmentBereavement leave | Fertility and growing family support | Flex Time Off | Health, dental, vision insurance | Life insuranceSenior-level Full TimeAmericas (USA or Canada) R1d ago
-
AI/ML Ops Engineer CAD 131K-164KAWS | Amazon Bedrock | Amazon ECS | Amazon EKS | Amazon SageMaker401k | Dental insurance | Discretionary time off | Health insurance | Life insuranceMid-level Full TimeRemote - Canada R1d ago
-
Analytics instrumentation | Automated testing | Backend APIs | CI/CD | Cloud infrastructureFoundational ownership | High autonomy | Remote workMid-level Full TimeCanada R2d ago
-
Generative AI Cloud Operations Engineer - Evinova CAD 134K-176KAWS | AWS CDK | Anthropic | Arize Phoenix | Azure Machine LearningHybrid work arrangementMid-level Full TimeCanada - Mississauga R2d ago
-
Senior Fabric Data Engineer CAD 114K-203KActive Directory | Audit Logging | Azure Active Directory | Azure DevOps | Data Governance100 percent remote work | Agile delivery | Fast paced team collaborationSenior-level Contract Full TimeToronto, Ontario, Canada - Remote R2d ago
-
Data Engineer CAD 61K-113KAWS Glue | Agile | Amazon Web Services | Apache Spark | Batch ProcessingAccident insurance | Career development support | Health insurance | Hybrid work schedule | Life insuranceMid-level Full TimeSCC, Canada R2d ago
-
Mid-level Full TimeSCC, Canada R2d ago
-
Active Learning | Calibration | Causal Inference | Constrained optimization | Contextual BanditsSenior-level Full TimeHybrid- Toronto, ON R4d ago
-
Machine Learning Engineer USD 150K-250KLanguage Processing | Machine Learning | Natural Language | Natural Language Processing | PyTorchOpen source portfolio | Publications in ML venues | Startup experience | Understanding model failures in the wild | Visa sponsorshipMid-level Full TimeToronto Hybrid R4d ago
-
Sr Analytics Developer, Finance, Brokerage & Market Risk CAD 151K-189KAWS | Amazon Redshift | Amazon S3 | Apache Airflow | DBTEmployer matched savings plan | Flexible sick leave | Health benefits | Life insurance | Paid vacationSenior-level Full TimeRemote (Canada) R4d ago
-
Senior MLOps Engineer II (AI Native) USD 148K-216KAWS | Apache Airflow | Apache Ray | Apache Spark | Azure Storage401k match | Employee assistance program | Health insurance | Mental wellness program | Paid parental leaveMid-level Full TimeRemote, USA ; Remote, Canada R4d ago
-
Data Engineer, DMP CAD 87K-108KApache Airflow | Apache Spark | BigQuery | Cloud Dataflow | Data ModelingBirthday day off | Course Reimbursement Program | Free online doctor visits | Hybrid work environment | Learning academyMid-level Full TimeOakville, Canada R4d ago
-
Data Engineer CAD 100K-170KAWS | Apache Airflow | Azure | BigQuery | Cloud Computing401k | Asynchronous work culture | Autonomy | Company paid life insurance | Disability coverageMid-level Full TimeRemote - US or Canada R4d ago
-
AWS | Access Control | Alerting | Audit Logging | AzureCollaborative environment | Fully remote option in Canada | Mentoring and leadership opportunities | Opportunity to shape organization wide machine learning and generative AI infrastructure strategySenior-level Full TimeCanada R5d ago
-
AWS | AWS CloudFormation | Alerting | Amazon Kinesis | Apache FlinkCollaborative environment | Comprehensive medical benefits | Financial benefits | Remote work (Canada) | Technical ownershipMid-level Full TimeCanada R5d ago
-
AWS | Access Control | Apache Airflow | Apache Flink | ClickHouseCollaborative work environment | Comprehensive medical benefits | Inclusive workplace | Remote work opportunity | Technical growth and learningSenior-level Full TimeCanada R5d ago
-
Senior Software Engineer - AI Interaction Evaluator (Codex / Claude Code, up to $200/hr) USD 200K-400KClaude Code | Cursor | JavaScript | OpenAI Codex | PythonSenior-level Full TimeMiami R5d ago
-
Agent Orchestration | Agent SDK | Automated testing | CI/CD | Claude Agent SDKHybrid work environmentMid-level Full TimeMontreal 700, Canada R5d ago
-
AWS Lambda | Amazon Bedrock | Amazon Q | Amazon Q Business | Amazon S3Hybrid work model | Mentorship | Professional training | Wellness programsSenior-level Full TimeSun Life Waterloo King, Canada R5d ago