AI QA Trainer - LLM Evaluation
Netherlands
R
A EUR 54K-75K (estimate) Senior-level Contract Full Time
Tasks
- Analyze model failures and failure patterns
- Assess hallucination and factual accuracy
- Automate evaluation using Python and SQL
- Collaborate on evaluation metrics dashboards and monitoring
- Design test plans and regression suites
- Develop evaluation frameworks and rubrics
- Evaluate large language models
- Maintain documentation and reproducible issue reports
- Perform adversarial testing and red teaming
- Recommend prompt and guardrail improvements
- Verify grounding and retrieval augmented generation outputs
Perks/Benefits
Skills/Tech-stack
Adversarial Testing | Bias Assessment | Experiment tracking | LLM Evaluation | Language Models | Language Processing | Large Language Models | Machine Learning | Natural Language | Natural Language Processing | Prompt engineering | Python | Red Teaming | Regression testing | Retrieval-Augmented Generation | SQL | Safety testing | Test automation | Test frameworks
Education
Related jobs
-
Data Scientist - Machine Learning EUR 18K-23KCI/CD | Cloud infrastructure | Cloud platform | Git | Google CloudExtra day off | Hybrid work | Learning and development initiatives | Mentorship program | Relocation packageMid-level Full TimeAmsterdam, Netherlands R1d ago
-
AI Governance | AWS | Agent systems | Cloud Architecture | Conversation memoryContract extension option | Flexible consulting environment | Fully remote | US time zone overlapSenior-level Contract Full TimeNetherlands R1d ago
-
Cloud Data | Cloud data warehousing | Data Contracts | Data Ingestion | Data LineageAutonomy | Collaborative team | Flexible working arrangements | Fully remote work | Professional growth opportunitiesSenior-level Full TimeNetherlands R1d ago
-
AI Agents | AI Observability | API Management | Access Control | Artificial IntelligenceAutonomy | Continuous growth culture | Corporate events access | Innovation culture | Medical insuranceExecutive-level Full TimeNetherlands R2d ago
-
API Design | APIs | Backend architecture | Command Line | Command-line InterfaceCompany-provided equipment | Continuous learning allowance | Flexible working hours | Fully remote | Health insurance allowanceSenior-level Full TimeNetherlands R3d ago
-
AWS | Bash | Cloud platform | Databricks | DatadogEquity package | Fully remote Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeNetherlands R4d ago
-
AWS | Access Control | Agent systems | Agentic AI | AzureEducation support | Flexible working arrangements | Leadership development | Learning programs | MentorshipSenior-level Full TimeVirtual Office (Netherlands) R4d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R4d ago
-
Alerting | Amazon Aurora | Amazon Redshift | Batch Processing | CI/CDFully remote workSenior-level Full TimeNetherlands R4d ago
-
Staff/Lead Python Engineer (FastAPI, Orchestration) EUR 82K-112KAsynchronous programming | Cloud platform | Concurrent programming | FastAPI | GenAIAnnual learning and development budget | Co-working budget | Company laptop | Enhanced parental leave | Flexible scheduleSenior-level Full TimeRemote Anywhere R5d ago
-
AWS | Big Data | Hadoop | Java | Machine LearningDocumentation support | Remote workSenior-level Full TimeAmsterdam, North Holland, Netherlands R7d ago
-
Forward Deployed AI Engineer EUR 54K-74KAgent Orchestration | Agent systems | Agentic AI | Artificial Intelligence | Data EngineeringCertification support | Coaching | Equipment provided | Flexible vacation days | Hybrid workMid-level Full TimeUtrecht, Utrecht, Netherlands R8d ago
-
Auto Scaling | Blue-Green Deployment | Blue/green | CI/CD | CUDAFully remote | High ownership and influence | Meaningful technical impact | Professional growth opportunities | Remote-first flexibilitySenior-level Full TimeNetherlands R8d ago
-
Cloud Data Engineer - Lead Consultant EUR 68K-86KCloud platform | Data Engineering | Google Cloud | Google Cloud Platform | PythonContinuous training and development | Hybrid work | Remote work 4 weeks per year | Team-building activitiesSenior-level Full TimeAmsterdam, NH, Netherlands R10d ago
-
Data Platform Engineer EUR 45K-57KAirflow | ArgoCD | Avo | CI/CD | DBTCollaborative team | Hybrid work setupMid-level Full TimeAmsterdam; Dublin; Helsinki HQ R11d ago
-
AI Coding Agents | AI coding | Agent Orchestration | Alerting | Benchmarking26 vacation days | E-learning support | Flexible working hours | Hybrid work | Language trainingSenior-level Full TimeNLD - Alphen Aan Den Rijn, … R16d ago
-
AI Engineer EUR 54K-75KDeep learning | Entity recognition | Explainable AI | GPU clusters | Hugging FaceCareer growth | Collaborative team culture | Conference budget | High end GPU access | Hybrid workSenior-level Full TimeNetherlands - Alkmaar R16d ago
-
Senior ML Engineer (Token Factory) GBP 75K-130KAttention | CI/CD | CUDA | Cutlass | FP8Career growth and learning opportunities | Collaborative and innovative culture | Flexibility | International environment | Opportunity to work on impactful AI projectsSenior-level Full TimeAmsterdam, Netherlands; Berlin, Germany; Israel; London, … R23d ago
-
Senior ML Engineer (Token Factory) GBP 80K-130KCI/CD | Distributed Training | Inference Optimization | JAX | JAX Speculative DecodingCareer growth and learning opportunities | Collaborative culture | Flexibility | International environment | OwnershipSenior-level Full TimeGermany; Israel; Netherlands; Prague, Czech Republic; … R23d ago
-
Staff ML Engineer EUR 56K-84KAWS | Amazon SageMaker | Apache Airflow | Apache Flink | CI/CDHome office stipend | Manager training | PTO | Paid parental leave | Professional development coursesSenior-level Full TimeNetherlands R24d ago
-
Experimental Quantum Engineer EUR 45K-45KCryogenics | Device Characterization | Dilution refrigerator | Microwave measurement | PythonHybrid work | Open vacation policy | Pension plan | Relocation support | Training budgetEntry-level Full TimeDelft R25d ago
-
Machine Learning Engineer EUR 69K-87KCloud Computing | Fraud Detection | Learning operations | Machine Learning | Machine Learning OperationsDigital Nomad program | Friday drinks | Hybrid work schedule | Learning budget | Lunch benefitsSenior-level Full TimeAmsterdam, Noord-Holland, Netherlands R30d ago
-
Lead Data Platform Architect 12 Months Contract EUR 53K-69KADaM | Access Control | Azure | Azure Data | Azure Data FactorySenior-level Contract Full TimeAmsterdam, North Holland, Netherlands - Remote R1mo ago
-
AI Solutions Engineer EUR 62K-90KAI Services | Azure AI | Azure AI Services | Feature Engineering | Fine TuningAnnual training courses | Bike leasing program | Extra pension savings option | Health and vitality offer | Hybrid work modelSenior-level Full TimeUtrecht, Utrecht, Netherlands R1mo ago
-
Airflow | Apache Kafka | BigQuery | Computer Vision | Data PipelinesCoaching benefits | Employer sponsored medical plan | Family planning support | Flexible vacation | Gender affirming care supportSenior-level Full TimeRemote - The Netherlands R1mo ago