Principal Engineer, AI Inference Reliability
US and Canada Offices
CAD 145K-217K (estimate) Senior-level Full Time
Tasks
- Architect reliability and observability
- Build dashboards and alerts
- Build reliability tooling
- Communicate reliability status
- Create chaos testing tools
- Define reliability strategy
- Design reliability mechanisms
- Establish SLOs
- Execute distributed fault injection
- Implement failover
- Implement fault detection
- Implement graceful degradation
- Implement recovery
- Implement throttling
- Lead incident management
- Mentor engineers
- Monitor reliability metrics
- Own postmortems
- Perform root cause analysis
- Run load simulation
- Run prevention loops
- Set reliability best practices
Perks/Benefits
- N/A
Skills/Tech-stack
Alerting | C++ | Chaos Engineering | Dashboards | Distributed Fault Injection | Distributed Systems | Failover | Fault detection | Fault injection | Go | Graceful Degradation | Incident Response | Load Testing | Monitoring | Observability | Postmortem | Python | Recovery | Rust | SLA | SLI | SLO | Throttling
Education
Related jobs
-
Embedded Firmware Engineer CAD 100K-120KBuild systems | C# | C++ | Driver BSP | Embedded LinuxFree on-site gym | Health and wellness facilities | Hybrid work after initial period | On-site daycare | On-site parkingMid-level TemporaryVancouver, BC, CA6h ago
-
Machine Learning Engineer - EA Sports FC CAD 92K-129K3D Animation | 3D transformations | Argo Workflows | Azure | C++Free games | Health insurance | Hybrid work arrangement | Paid time off | Retirement savingsMid-level Full Time地点：Vancouver, British Columbia, Canada8h ago
-
API Development | Backend Development | Cloud platform | Data Processing | Distributed SystemsBonus | Equity | Health benefits | Professional developmentSenior-level Full TimeWaterloo, ON, Canada; Toronto, ON, Canada9h ago
-
Data Science & AI CAD 71K-106KApache Spark | Backtesting | Calibration | Cross-validation | Data GovernanceDental insurance | Disability insurance | Employee assistance program | Life insurance | Medical insuranceMid-level Full TimeDorval, Québec, CA, H4S 1Y9 R15h ago
-
CUDA | DVC | Distributed Systems | FastAPI | Feature StoreEmployee resource groups | Employer Matched Savings Program | Equity compensation | Flexible sick days | Health benefitsSenior-level Full TimeCanada R17h ago
-
Senior Software Engineer, Datastores (Auth0) CAD 136K-187KAmazon DynamoDB | Amazon Web Services | CI/CD | Go | KubernetesDental insurance | Health insurance | Healthcare spending account | Paid leave | Paid time offSenior-level Full TimeToronto, Ontario, Canada R20h ago
-
Jr Data Engineer CAD 125K-150KAirflow | Apache Airflow | BigQuery | Cloud Composer | Cloud RunClear internal promotion pathways | Generative AI tools access | Hybrid work | In office days per week 3 plus | In role coachingMid-level Full TimeToronto, Ontario, Canada R21h ago
-
Junior Analytics Engineer CAD 50K-65KAirflow | Automated testing | CTE | DBT | Data ModelingFlexible PTO policy | Health benefits | Professional learning and development | Remote-first work environmentEntry-level Full TimeCanada R1d ago
-
Data Science Manager, Risk CAD 90K-130KData Pipelines | Decision Tree | Feature Engineering | Loss forecasting | Machine LearningContinuing education stipend | Generous time off | Health and Wellness Spending Accounts | Hybrid work environment | Medical/Dental/Vision insuranceMid-level Full TimeToronto, Ontario R1d ago
-
Senior AI/ML Software Engineer CAD 150K-180KAPI Security | AWS | Agile | Apache Spark | Data Pipelines401k | Dental insurance | Discretionary time off | Health insurance | Life insuranceSenior-level Full TimeRemote - Canada R1d ago
-
AI/ML Software Engineer CAD 128K-158KAPI Security | AWS | Agile | Apache Spark | Authentication401k | Dental insurance | Discretionary time off | Health insurance | Life insuranceMid-level Full TimeRemote - Canada R1d ago
-
Security and Infrastructure Engineer CAD 120K-150KAWS | AWS Well Architected | Agentic AI | Azure | CIEMComprehensive benefits | Employee assistance plan | Flexible work schedule | Learning and development budget | Remote-firstSenior-level Full TimeRemote - Canada R1d ago
-
Staff Software Engineer, Product Risk CAD 172K-220KAmazon Web Services | Apache Airflow | Apache Flink | Apache Hive | Apache Hive MetastoreSenior-level Full TimeToronto1d ago
-
Machine Learning Engineer II (Underwriting ML) CAD 133K-183KAirflow | Backtesting | CatBoost | Classification | DaskDental coverage | Employee stock purchase plan | Flexible spending accounts | Remote work | Subsidized medical coverageMid-level Full TimeRemote Canada R1d ago
-
Research Software Engineer (hybrid Montreal) CAD 150K-189KBGP | DNS | Go | HTTP | HTTPSHybrid work environmentSenior-level Full TimeOld Montreal, Canada1d ago
-
AI Solutions Engineer CAD 85K-114KAPI Integration | ChatGPT | Claude | CrewAI | Data PipelinesEmployee discounts | Group benefits | Paid time off | RRSP matchMid-level Full TimeHybrid- Toronto, ON R1d ago
-
Senior Machine Learning Engineer CAD 140K-180KA/B | A/B Testing | Apache Spark | B testing | Cloud platformFinancial benefits | Medical benefitsSenior-level Full TimeMovable Ink - Toronto (Remote) R1d ago
-
Applied Intuition | Applied Intuition Simian | C++ | CI/CD | CMakeHybrid work | Medical/Dental/Vision insurance | Personal days | RRSP program | Sick daysMid-level Full TimeMontréal, QC, Canada R1d ago
-
AWS | AWS CloudFormation | AWS Glue | AWS Glue Catalog | AWS LakeCompetitive contract compensation | Flexible work schedule | Fully remote work | Technical autonomySenior-level Contract Full TimeCanada R1d ago
-
AlloyDB | Backup and Restore | Bash | CDC | Cloud BigtableLarge scale transformation projects | Professional growth opportunities | Remote work flexibilitySenior-level Full TimeCanada R1d ago
-
DBT | Data Governance | Data Lineage | Data Modeling | Data PipelinesProfessional development | Reasonable accommodations during hiring and employment | Remote work flexibility within Canada | Supportive learning culture | Work Eastern Time hoursMid-level Full TimeCanada1d ago
-
AI Agents | Alerting | C# | Event Driven | Event-driven architectureContinuous learning | Fully remote work | Professional growthMid-level Full TimeCanada R1d ago
-
Agile | Azure Data | Azure Data Factory | Azure Data Services | Azure SynapseContract opportunity | Fully remote CanadaMid-level Contract Full TimeCanada R1d ago
-
Data Engineer CAD 108K-135KAmazon S3 | Apache Airflow | Apache Hadoop | Apache Hive | Apache SparkChild care benefits | Disability benefits | Family building benefits | Health and dental insurance | Health care savings accountMid-level Full TimeToronto, Canada R1d ago
-
Senior-level Full TimeMarkham, Ontario, Canada1d ago