Senior Quality Engineer, Data Science
16 YORK ST:TORONTO, Canada
CAD 75K-140K (estimate) Senior-level Full Time
Tasks
- Analyze outputs debug issues and propose corrective actions
- Assess hallucination risk and business alignment
- Automate testing validation and monitoring processes
- Collaborate with business SMEs on objectives and risk tolerances
- Collaborate with data scientists on data pipelines feature engineering models and prompts
- Create statistical robustness and performance tests
- Define track and report data quality and model quality metrics
- Design validation tests for machine learning models and GenAI solutions
- Develop automated evaluation frameworks
- Evaluate LLM outputs for accuracy consistency and bias
- Test guardrails prompt strategies fallback mechanisms and failure handling
- Validate agent workflows and decision paths
- Validate solutions using Python and related tools
Perks/Benefits
Skills/Tech-stack
A/B | A/B Testing | Agile | Apache Spark | B testing | Data Drift | Data Pipelines | Data Quality | Experimental Design | Explainability | Fairness | GenAI | Governance | Hadoop | Information security | Language Models | Large Language Models | Machine Learning | Model Risk Management | Model risk | Monitoring | Prompt engineering | Python | Quality Assurance | Risk Management | Statistical Testing | Test automation
Education
Related jobs
-
Staff AI Engineer (Global Security) CAD 90K-140KAWS | Anthropic Claude | Anthropic Claude API | Anthropic SDK | AzureAccess to challenging work | Bonuses | Career coaching and development | Flexible benefits | Flexible work life balance optionsSenior-level Full Time16 YORK ST:TORONTO, Canada20h ago
-
AI Engineer CAD 82K-151KAPI Design | API Gateway | AWS SageMaker | Anthropic Claude | Audit LoggingFlexible work-life balance | Training programSenior-level Full TimeRBC WATERPARK PLACE, 88 QUEENS QUAY …20h ago
-
Lead QE AI Engineer CAD 90K-141KAPI Testing | Agile methodology | Artificial Intelligence | CI Pipeline | CI/CDSenior-level Full Time1597 BEDFORD HWY:HALIFAX, Canada20h ago
-
Principal AI Engineer CAD 90K-154KAPI Design | API Gateway | JWT | Language Processing | Machine LearningCoaching development support | Collaborative high performing team | Flexible work-life balance | Opportunities to do challenging work | Opportunity for lasting impactSenior-level Full TimeRBC WATERPARK PLACE, 88 QUEENS QUAY …20h ago
-
Senior Software Engineer, Data Platform USD 163K-247KAWS | Amazon EMR | Amazon Kinesis | Amazon MSK | Amazon RedshiftHybrid workSenior-level Full TimeDenver, CO;San Francisco, CA;New York, NY;Los … R22h ago
-
Data Developer - TG Quality Engineering TGQF CAD 85K-107K.NET | AWS | Agile | Apache Flume | Apache NiFiSenior-level Full TimeQuebec City, QC, Canada1d ago
-
Data Engineer USD 133K-182KAWS | Airbyte | Airflow | Alerting | CI/CD401k contribution | Dental insurance | Health insurance | Life insurance | Long-term disabilitySenior-level Full TimeCanada - Remote R1d ago
-
Senior Data Engineer CAD 120K-145KApache Hadoop | Apache Kafka | Apache Spark | Azure Fabric | CI/CDFlexible working hours | Health and wellness benefits | Learning and development | Parental leave | Retirement planningSenior-level Full TimeToronto1d ago
-
Data Architect CAD 120K-150KA/B | A/B Testing | API Design | API Development | Amazon RedshiftAnnual tech allowance | Annual travel allowance | Flexible working hours | Health insurance | Learning and developmentSenior-level Full TimeToronto1d ago
-
DSP | FPGA | FPGA Integration | FPGA implementation | Hardware DesignDental insurance | Employee assistance program | Employer matching retirement savings | Flexible work hours | Health insuranceMid-level Full TimeMontreal (CAN), Canada1d ago
-
Data Engineer Co-op (8-12 Months) CAD 45K-45KAirflow | BI Dashboards | CI/CD | Cloud platform | Data ModelingCorporate partner discounts | Employee purchase program | Free Kobo device | Free weekly ebooks or audiobooks | Hybrid work modelNone Full TimeON-John Street, Canada1d ago
-
Senior-level Full Time661 University Avenue, Toronto, Ontario, Canada1d ago
-
Machine Learning Engineer II CAD 154K-199KAzure | Cloud Architecture | Data Engineering | Deep learning | Generative AIBanking benefits and discounts | Career development | Health and well-being benefits | Mentoring programs | Paid time offSenior-level Full Time661 University Avenue, Toronto, Ontario, Canada1d ago
-
GenAI Testing CAD 90K-115KAgile | BDD | CI/CD | Cucumber | End to EndCompany-Paid Holidays | Disability insurance | Employee assistance program | Life insurance | Medical, dental & vision coverageMid-level Full TimeToronto, CA1d ago
-
Expert MLOps Platform Engineer CAD 108K-214KAWS SageMaker | Apache Kafka | ArgoCD | Azure Event | Azure Event HubsSenior-level Full TimeVancouver, British Columbia, CA, V6B 1A91d ago
-
QA Data Engineer CAD 94K-118KAirflow | Amazon Web Services | Apache Spark | Azure | Azure DataCareer progression | Company pension plan | Development opportunities | Free parking | Team member offersMid-level Full TimeLangley Office - Langley BC, Canada1d ago
-
Senior Data Quality Automation Engineer CAD 100K-106KAnomaly Detection | CI/CD | Data Quality | Data Validation | ETLEmployee discounts | Flexible health benefits | Matched pension plan | RRSP program | TFSA programSenior-level Full Time00000 Canadian Headquarters, Canada1d ago
-
Lead Data Engineer (Azure / Databricks / Python) CAD 115K-165KAzure Data | Azure Data Factory | Azure DevOps | Azure Functions | CI/CDCollaborative environment | Impactful work | Innovative culture | Professional growthSenior-level Full TimeVancouver, British Columbia, Canada1d ago
-
Data Engineer (Azure / Databricks / Python) CAD 115K-163KAzure Data | Azure Data Factory | Azure DevOps | Azure Functions | CI/CDCollaborative environment | Impactful work | Innovative culture | Professional growthSenior-level Full TimeVancouver, British Columbia, Canada1d ago
-
Senior-level Full TimeToronto, Canada1d ago
-
Senior-level Full TimeToronto, Canada1d ago
-
Senior-level Full TimeToronto, Canada1d ago
-
Senior-level Full TimeToronto, Canada1d ago
-
Lead AI Engineer CAD 82K-141KAWS | Agent systems | Ansible | Anthropic Claude | AutogenCollaborative high performing team | Comprehensive total rewards program | Flexible work life balance options | Opportunities for greater accountabilities | World class training programSenior-level Full TimeRBC WATERPARK PLACE, 88 QUEENS QUAY …1d ago
-
Machine Learning & Data Engineer CAD 82K-135KAccess Control | CI/CD | Data Governance | Data Modeling | Data orchestrationCoaching and development support | Dynamic collaborative team | Flexible work life balance options | Opportunities to make impactSenior-level Full Time1 PLACE VILLE MARIE:MONTRÉAL, Canada1d ago