AI QA Trainer - LLM Evaluation
A CHF 128K-192K (estimate) Senior-level Contract Full Time
Tasks
- Analyze model failures and document root cause
- Assess safety and reliability
- Automate testing using Python or SQL
- Collaborate on evaluation metrics dashboards and monitoring
- Conduct hallucination detection tests
- Create evaluation frameworks
- Design test plans and regression suites
- Develop evaluation rubrics and benchmarks
- Evaluate large language models
- Maintain reproducible issue reports
- Perform adversarial testing and red teaming
- Provide recommendations for prompts guardrails and evaluation strategies
- Validate tool usage accuracy
- Verify grounding and retrieval augmented generation outputs
Perks/Benefits
Skills/Tech-stack
Adversarial Testing | Bias Assessment | Evaluation Frameworks | Experiment tracking | Guardrails | Hallucination detection | LLM Evaluation | Machine Learning | Prompt engineering | Python | Quality Benchmarks | Red Teaming | Regression testing | Retrieval-Augmented Generation | Risk Analysis | SQL | Software testing | Test automation
Education
Roles
AI | AI QA | AI QA Trainer | Evaluation Specialist | LLM Evaluation Specialist | QA Trainer | Specialist | Trainer
Related jobs
-
AI Agents | AI Foundry | Azure AI | Azure AI Foundry | Azure DevOpsFlexible working hours | Hybrid working | International working environment | Knowledge sharing sessions | Learning opportunitiesEntry-level InternshipZurich, Switzerland R16h ago
-
Cloud Computing | Code review | Data Modalities | Deep learning | DockerCollaborative & Inclusive Culture | Four-day work week | Fully remote work | Opportunity for open source contributions | Unlimited paid time offSenior-level Full TimeSwitzerland R22h ago
-
AI Governance | AWS | Agent systems | Amazon Bedrock | Cloud ComputingContract with extension option | Fully remote | US time zone overlapSenior-level Contract Full TimeSwitzerland R1d ago
-
AWS | Apache Airflow | Cloud platform | Deep learning | DockerAccess to modern development tools | Equity opportunities | Medical/Dental/Vision insurance | Unlimited paid time off | Work from homeMid-level Full TimeSwitzerland R7d ago
-
AWS | Apache Spark | Data Analysis | Hadoop | JavaRemote workSenior-level Full TimeZürich, Switzerland R7d ago