Overseas Contractor
Tasks
- Assess outputs for bias fairness and compliance
- Build and maintain automated test suites for AI pipelines
- Conduct adversarial testing for vulnerabilities like prompt injection and jailbreaks
- Design and execute test cases for AIML models
- Develop and maintain prompt test libraries
- Document risk findings and collaborate on remediation
- Evaluate LLM responses for relevance tone factual accuracy and business rule adherence
- Identify hallucinations response drift and edge case failures
- Integrate AI testing into CI/CD workflows
- Perform regression testing after model updates retraining or data changes
- Track and report quality metrics performance benchmarks and defect trends
- Validate model outputs for accuracy consistency and expected behavior
- Write test plans defect reports and evaluation summaries
Perks/Benefits
- N/A
Skills/Tech-stack
Adversarial Testing | Bias Testing | CI/CD | Generative AI | LLM Testing | Language Models | Language Processing | Large Language Models | Machine Learning | NLP | Natural Language | Natural Language Processing | Prompt engineering | Python | Quality metrics | Regression testing | Responsible AI | Test automation
Education
N/A
Roles
AI | AI Tester | Engineer | QA Engineer | Software QA | Software QA Engineer | Tester
Related jobs
-
Forward Deployed Developer, GenAI, Google Cloud CAD 114K-117KAPI Integration | APIs | Agent systems | Cloud platform | Cost Per RequestSenior-level Full TimeToronto, ON, Canada; Montreal, QC, Canada1h ago
-
Senior Forward Deployed Developer, GenAI, Google Cloud CAD 170K-174KAPIs | Accuracy | Agent systems | Cloud platform | Cost Per RequestSenior-level Full TimeToronto, ON, Canada; Montreal, QC, Canada1h ago
-
Machine Learning Engineer II (Fraud) CAD 125K-175KAirflow | Apache Spark | Backtesting | CatBoost | DaskDental and vision coverage | Employee stock purchase plan | Health coverage | Paid time off | Remote workMid-level Full TimeRemote Canada R13h ago
-
Computer Vision | Data Analysis | Deep learning | Ecommerce | Fine TuningFlexible schedule | Hybrid work | Remote work up to 4 weeks per yearSenior-level Full TimeToronto, ON R14h ago
-
Machine Learning Engineer CAD 122K-183KAd Ranking | Airflow | Apache Spark | BigQuery | Convolutional Neural Network401k employer match | Caregiving support | Comprehensive healthcare benefits | Family planning support | Flexible vacationMid-level Full TimeRemote - Ontario, Canada R16h ago
-
AWS | Anomaly Detection | C# | C++ | CsharpDental insurance | Disability insurance | Flexible work schedule | Health insurance | Life insuranceMid-level Full TimeQuébec, Qc16h ago
-
Mid-level Full TimeMontréal, QC, Canada16h ago
-
Mid-level Full TimeToronto, ON, Canada17h ago
-
Applied Scientist CAD 145K-172KGit | Information Retrieval | Knowledge Base | Knowledge Base Retrieval | Language ModelsMid-level Full TimeKitchener, Canada18h ago
-
Specialist, Data Engineering CAD 81K-116KAzure Data | Azure Data Factory | Azure Databricks | Azure SQL | Azure SQL DatabaseCareer development | Flexible benefits plan | Flexible savings fund option | Home internet allowance | Hybrid work optionMid-level Full TimeSherbrooke, Quebec, Canada; Montréal, Quebec, Canada R19h ago
-
Data Analytics Engineer CAD 115K-149KAmazon Web Services | Azure Data | Azure Data Lake | Cloud platform | ComplianceDEI initiatives | Dental insurance | Medical insurance | Mental health support | Retirement plansSenior-level Full TimeRemote, Canada R20h ago
-
AWS Glue | AWS Lambda | Amazon S3 | Azure Data | Azure Data FactoryBilingual working environment | Contract renewal possibilitySenior-level ContractEst de Montréal, Canada23h ago
-
Staff Software Developer, AI/ML, Safety and Security USD 207K-300KComputer Vision | Data Processing | Debugging | Deep learning | Fine TuningSenior-level Full TimeWaterloo, ON, Canada; New York, NY, …1d ago
-
Agile | Alerting | Apache Airflow | Automation | BigQueryCareer growth and development | Community involvement opportunities | Health and wellbeing resources | Hybrid work with at least 3 days in office | Paid sick daysSenior-level Full TimeToronto, ON, M2N 5M9, CA1d ago
-
3D Vision | BEV Modeling | Cloud processing | Computer Vision | Deep learningDental insurance | Flexible-hybrid work | Holiday closure | Life insurance | Medical insuranceSenior-level Full TimeCanada1d ago
-
Director of AI Engineering – Generative AI & Autonomous Systems (10033) Toronto, Canada CAD 166K-220KAWS | Agent systems | Artificial Intelligence | Azure | Cloud NativeExecutive-level Full TimeToronto, Canada1d ago
-
Member of Technical Staff (Applied AI Engineer) CAD 140K-260KAPI Design | Browser Automation | CSS | GraphQL | HTMLDental insurance | Gym membership subsidy | Health insurance | Offsites | Premium laptopSenior-level Full TimeToronto, Ontario, Canada1d ago
-
C# | MATLAB | NumPy | Pandas | PythonFlexible hours | Part-time availability | Project based workSenior-level Full TimeCanada - Remote R1d ago
-
Customer Analytics | Data Analysis | Data Ingestion | Data cleaning | Exploratory Data AnalysisPart-time flexible hours | Project based workMid-level FreelanceCanada - Remote R1d ago
-
Freelance Machine Learning Engineer CAD 110KLangchain | MLOps | NumPy | Pandas | Prompt engineeringEnglish documentation requirement | Flexible hours | Project based workMid-level FreelanceCanada - Remote R1d ago
-
Langchain | Language Models | Large Language Models | MLOps | NumPyFlexible workload | Part-time schedule | Project based workMid-level FreelanceCanada - Remote R1d ago
-
Data and Analytics - Data Analysis, Senior Associate CAD 84K-134KAWS | Azure | BERT | CI/CD | ContainerizationFlexible work arrangements | Hybrid work environment | Inclusive benefits | Wellbeing support programsSenior-level Full TimeToronto - 18 York Street, Canada1d ago
-
Applied AI ML Engineer - Ottawa, ON CAD 105K-130KAWS | Apache Spark | Azure | CUDA | GCPEmployee resource groups | Health and dental coverage | Life insurance | Long-term disability | Mental health wellness programMid-level Full TimeOttawa, Canada1d ago
-
Applied AI ML Engineer - Ottawa, ON CAD 76K-105KAWS | Agent systems | Apache Spark | Azure | CUDAAnnual holidays | Critical illness insurance | Employee resource groups | Health and dental coverage | Life insuranceMid-level Full TimeOttawa, Canada1d ago
-
Senior Machine Learning Engineer, vLLM CAD 130K-200KComputer Vision | Deep learning | Graph theory | Inference Optimization | LLM InferenceOpen source collaboration culture | Remote work flexibilitySenior-level Full TimeRemote CA ON, Canada R1d ago