Distributed Inferencing Software Engineer - AI Models
Tasks
- Analyze and optimize throughput and low latency
- Build distributed model management monitoring benchmarking documentation
- Develop parallelization strategies for AI workloads
- Enable benchmark AI models on distributed systems
- Optimize distributed workloads for scale-up multi GPU scale-out multi node
Perks/Benefits
- N/A
Skills/Tech-stack
C++ | CUDA | DeepEP | Expert Parallelization | HIP | KV cache | Kubernetes | Llama.cpp | OpenCL | Python | SGLang | Slurm | VLLM
Education
Related jobs
-
Principal Software Development Engineer, MLOps CAD 168K-252KAWS | Access Control | Amazon SageMaker | CI/CD | Cloud EngineeringSenior-level Full TimeCanada, ON, Toronto16h ago
-
Senior Machine Learning Engineer, Growth CAD 170K-190KAWS | Azure | CI/CD | Data Pipelines | Drift DetectionFitness membership discount | Generous vacation and PTO | Health & dental benefits | Health spending account | Learning and development supportSenior-level Full TimeToronto, Ontario, Canada17h ago
-
Sales Engineer (AI & Data Platforms) CAD 140K-160KAPIs | Agent-based | Agent-based systems | Dashboards | Data ArchitectureContinuous learning opportunities | Dental insurance coverage | Health insurance coverage | Hybrid work schedule | Vision insurance coverageSenior-level Full TimeToronto, Ontario20h ago
-
Software Engineer, Data Pipelines CAD 108K-135KAWS | Ansible | Chef | CloudFormation | DockerChild care benefits | Dental insurance | Disability benefits | Family building benefits | Flexible paid time offMid-level Full TimeToronto, Canada21h ago
-
Staff Applied AI Engineer USD 180K-273KAWS | Agentic AI | Arize | Bedrock | CI/CD401k company match | Fertility assistance | Flexible time off | Medical, dental & vision coverage | Mental health programSenior-level Full TimeRemote - US & Canada R21h ago
-
Senior Machine Learning Engineer, Ads CAD 156K-203KAlerting | Apache Kafka | Apache Spark | Distributed Systems | Feature EngineeringCaregiving support | Comprehensive medical benefits | Family planning support | Flexible vacation | Gender-affirming careSenior-level Full TimeRemote - Ontario, Canada R23h ago
-
Machine Learning Engineer, Ads CAD 148K-203KApache Kafka | Apache Spark | Data Processing | Distributed Systems | Feature EngineeringCaregiving support | Comprehensive medical benefits | Family planning support | Flexible remote work | Gender-affirming careSenior-level Full TimeRemote - Ontario, Canada R23h ago
-
Senior-level Full TimeOttawa, Ontario, Canada1d ago
-
Access Control | Agile | Azure | Azure Data | Azure Data FactoryFull time 37 50 hours per week | Hybrid on site Tuesday to Thursday | Remote work Monday and FridaySenior-level ContractToronto, Canada1d ago
-
A/B | A/B Testing | B testing | Context Management | EmbeddingsEquity participation | Fully remote | Healthcare coverage | Home office setup allowance | International company retreatsSenior-level Full TimeCanada R1d ago
-
Agentic Solution Engineer CAD 120K-145KAI Governance | AI orchestration | AWS | Azure | Data integrationMid-level Full TimeToronto , Canada1d ago
-
Agentic Senior Solution Engineer CAD 110K-180KAI Governance | AI orchestration | AWS | Data Flow | GCPSenior-level Full TimeToronto , Canada1d ago
-
Senior Staff Developer, Machine Learning CAD 118K-159KArgo | CI/CD | Deep learning | Distributed Computing | DockerFlexible vacation | Flexible work options | Hackathons | Mentorship programs | Recognition programsSenior-level Full TimeRemote, Canada R1d ago
-
Data Engineer (GCP) CAD 90K-119KAirflow | Apache Beam | Apache Spark | Atlassian | Azure DataFlexible remote work within Canada | Permanent Canadian residentMid-level Full TimeToronto, Canada1d ago
-
Algorithms | C++ | Data Structures | Design Patterns | JavaMid-level Full TimeVancouver, British Columbia, CAN1d ago
-
Data Engineer CAD 100K-110KAzure Data | Azure Data Factory | Azure DevOps | Azure DevOps Services | Azure StorageMid-level Full TimeToronto, CA1d ago
-
AI Solutions Engineering Specialist (Hybrid) CAD 90K-154KAWS | Angular | Azure | Big Data | C#Career development opportunities | Employee benefits | Hybrid work environmentSenior-level Full TimeMontreal 700, Canada1d ago
-
Software Engineer CAD 110K-166KAWS | Automated testing | Azure | CI/CD | CSSDirect customer impact | Fast-paced startup environment | Travel opportunitiesMid-level Full TimeToronto, Ontario, Canada - Remote R1d ago
-
Lead Software Engineer, AI CAD 140K-175KAI Governance | APIs | AWS | Agentic tools | AzureAccess to Headspace app | Career development and growth | Employee incentive programs | Flexible vacation | Flexible work-life balanceSenior-level Full TimeCanada, Toronto, Ontario1d ago
-
Senior Data Engineer CAD 130K-180KCCPA | CDC | CI/CD | Compliance | Data Governance401k | Dental insurance | Health insurance | PTO | Vision insuranceSenior-level Full TimeMontreal, Toronto1d ago
-
Entry-level Full TimeCanada1d ago
-
Applied AI/ML Scientist USD 116K-159KA/B | A/B Testing | AWS | Anthropic | AzureCharitable matching | Childcare and eldercare support | Dental insurance | Disability insurance | Financial wellness supportMid-level Full TimeRemote Canada R1d ago
-
ADC Interface | Adaptive algorithms | Analog control | Analog control loops | Bare MetalDiscretionary bonus | Equity package | Health, dental, vision coverage | Professional development opportunities | Remote workSenior-level Full TimeCanada - Remote R1d ago
-
System Integration Specialist - Robotics CAD 92K-115KAutomation | Customer communication | Hardware Integration | Project Management | PythonGroup benefits plan | Hybrid work | Mentorship program | Paid volunteering days | Training and professional developmentMid-level Full TimeMontreal, QC, Canada1d ago
-
Staff ML Engineer CAD 196K-269KA/B | A/B Testing | AWS | Apache Spark | B testingComprehensive health insurance | Parental leave plans | Professional development stipend | Remote workSenior-level Full TimeRemote - Canada R1d ago