AI Engineer – AI Observability and Quality
Tasks
- Build automated test suites
- Create evaluation frameworks
- Define evaluation standards and practices
- Design evaluation infrastructure
- Develop regression detection systems
- Measure response accuracy
- Monitor cost and latency
- Operate independently and set roadmap
- Translate quality criteria into test scenarios
- Validate citation correctness
Perks/Benefits
- Company paid cell phone plan
- Dental insurance
- Employee assistance program
- Flexible work hours
- Health insurance
- Life insurance
- Long-term disability insurance
- New Parenting Leave
- Paid sick leave
- Paid vacation days
- Part-time option
- Remote work option
- Retirement savings plan with match
- Second medical opinion
- Technology allowance
- Vision insurance
Skills/Tech-stack
A/B | A/B Testing | B testing | DeepEval | Evaluation Frameworks | Generative AI | Information Retrieval | LLM Evaluation | Langfuse | Langsmith | Language Models | Large Language Models | Machine Learning | Python | RAG | Ragas | Significance Testing | Statistical significance | Statistical significance testing | Weights and Biases
Education
N/A
Related jobs
-
NumPy | Numerical Simulation | Pandas | Physics modeling | PythonFlexible schedule | Part-time project-based work | Remote workSenior-level FreelanceCanada - Remote R1d ago
-
Mid-level Full TimeMontreal, QC, Canada1d ago
-
Sr. Embedded & Compute Software Developer USD 130K-160KC# | C++ | CUDA | DO-178 | Debugging401k matching | Dental insurance | Employee assistance program | Health insurance | Paid HolidaysSenior-level Full TimeRemote (United States); Canada R1d ago
-
Process Automation Engineer CAD 80K-113KAgile | Automation | Cloud platform | Dashboards | Data PipelinesAssociate discount | Health and dental benefits | Hybrid work | Learning and development programs | Performance bonusesMid-level Full TimeRichmond Hill, ON, Canada1d ago
-
AWS Lambda | Anthropic | Authentication | BigQuery | ChromaDBAnnual leave | Company RSUs | Developer productivity support | Remote workSenior-level Full TimeCanada (Remote) R1d ago
-
Staff, Machine Learning Engineer CAD 130K-170KCI/CD | Evaluation | Grounding | Langchain | LanggraphCompany-provided benefits | Flexible PTO | Fullscript discounts | HSA | Hybrid/Remote flexibilitySenior-level Full TimeOttawa, ON1d ago
-
Senior-level Full TimeCA-ON-Mississauga1d ago
-
Senior-level Full TimeCA-ON-Mississauga1d ago
-
Gen AI Developer CAD 125K-145KAPIs | Agentic Frameworks | Anthropic Claude | FastAPI | Google GeminiSenior-level Full TimeCA-ON-Mississauga1d ago
-
Azure Data | Azure Data Factory | C# | Data Factory | Data GovernanceContract employment | Hybrid work | Onsite visits weeklySenior-level Full TimeMontreal, QC, Canada1d ago
-
Data Contracts | Data Governance | Data Pipelines | DataOps | MCPContract employment | Hybrid work scheduleSenior-level Full TimeMontreal, QC, Canada1d ago
-
Expert Embedded Software -DO178 CAD 105K-140KARINC 429 | C# | CAN bus | Configuration Management | DO-178BFlexible work location and schedule | Hybrid work model | Medical/Dental/Vision insurance | Personal leave | RRSP programSenior-level Full TimeMontréal, QC, Canada1d ago
-
AI | AI frameworks | Acceleration Libraries | C# | C++Senior-level Full TimeMarkham, Ontario, Canada1d ago
-
AI acceleration | AI accelerators | Artificial Intelligence | C# | C++Senior-level Full TimeMarkham, Ontario, Canada1d ago
-
Distinguished Engineer - AI Computing System CAD 172K-230KArtificial Intelligence | Cloud Computing | Cluster computing | Cluster scheduling | Co-designPermanent role | Research collaboration opportunities | Team leadershipExecutive-level Full TimeMarkham, Ontario, Canada1d ago
-
AI/ML Systems Design Engineer CAD 108K-159KArtificial Intelligence | Communication | Data Science | Documentation | Machine LearningMid-level Full TimeMARKHAM, ON, Canada1d ago
-
Agile | Apache Spark | Data Engineering | Design Patterns | JavaMid-level Full TimeToronto, Ontario, CAN2d ago
-
Chemistry & Python Expert - Freelance AI Trainer CAD 70K-70KC# | Computational Chemistry | MATLAB | Molecular Modeling | NumPyProject based workSenior-level FreelanceCanada - Remote R2d ago
-
Senior Engineering Lead, Applied Intelligence CAD 112K-140KAKS | API Integration | Agentic Systems | Angular | AuthenticationDefined benefit pension plan | Employee discount programs | Employer paid benefits coverage | Flexible Travel Program | Flexible work environmentSenior-level Full TimeToronto - 36.25, Canada2d ago
-
Manager, Data Analytics & Model Optimization - Global Compliance Trade Surveillance (Contract) CAD 81K-136KAudit response | Automation | Business Intelligence | Capital Markets | CommodityMid-level ContractToronto, ON, CA, M5H1A12d ago
-
Lead Data Engineer CAD 94K-171KAgile | Data Modeling | Data Quality | Data Warehousing | ETLCommunity engagement opportunities | Employee benefits | Flexible vacation | Personal days | Sick daysSenior-level Full TimeToronto, ON, CA, M5H4A62d ago
-
Databricks/Fabrics Expert CAD 160K-180KBest practices | CI/CD | Databricks | Deployment Pipelines | GenAIHybrid workSenior-level Full TimeToronto, ON, CA, M5J 2P12d ago
-
Databricks/Fabrics Expert CAD 160K-180KCI/CD | Databricks | GenAI | Machine Learning | Microsoft FabricHybrid work environmentSenior-level Full TimeToronto, ON, CA, M5J 2P12d ago
-
Mid-level Full TimeCAN - Richmond, Canada2d ago
-
Data Scientist Intern CAD 52K-52KBigQuery | Clustering | Data Modeling | Data Warehousing | Machine LearningCasual dress code | Dog-friendly office | Monthly team events | Remote work permitted | SkyTrain nearbyEntry-level InternshipVancouver, British Columbia, Canada2d ago