AI QA Trainer - LLM Evaluation
Tasks
- Analyze model behavior and recommend improvements
- Assess hallucination detection and factual accuracy
- Automate testing using Python and SQL
- Collaborate on quality monitoring dashboards
- Create evaluation rubrics and quality benchmarks
- Design evaluation frameworks
- Evaluate large language models
- Execute test plans and regression suites
- Identify and document model failures
- Maintain reproducible issue reports and documentation
- Perform adversarial testing and red teaming
- Verify retrieval augmented generation outputs
Perks/Benefits
Skills/Tech-stack
Adversarial Testing | Bias Assessment | Experiment tracking | Factual consistency | Hallucination detection | LLM Evaluation | Language Models | Large Language Models | Machine Learning | Prompt engineering | Python | RAG | Red Teaming | Regression testing | Retrieval-Augmented Generation | SQL | Safety testing | Software testing | Test automation
Education
Roles
AI | AI QA | AI QA Engineer | Engineer | Learning Engineer | Machine Learning Engineer | QA Engineer
Related jobs
-
Cloud Computing | Data Pipelines | Docker | GPU | KubernetesFour-day work week | Fully remote | Open source opportunities | Unlimited paid time offSenior-level Full TimeFrance R23h ago
-
Lead Data Scientist (H/F) EUR 46K-61KAzure | Docker | Evaluation Frameworks | GCP | GitAccess to employee council | Bonus plan | Health insurance | Meal tickets | Team eventsSenior-level Full TimeCourbevoie, La Défense, Île-de-France, France R1d ago
-
AWS | Agent systems | Artificial Intelligence | Cloud Computing | Cloud platformContract extension opportunity | Fully remote | Innovation-focused projects | Remote consulting environment | US time zone overlapSenior-level Contract Full TimeFrance R1d ago
-
Backfilling | Cloud | Cloud Data | Cloud Data Warehouse | Data ContractsFlexible working arrangements | Fully remote across Europe | High autonomy | Meaningful overlap with international teams | Professional growth opportunitiesSenior-level Full TimeFrance R1d ago
-
AI Agents | AI FinOps | AI Observability | AI systems | API IntegrationCorporate events | Fully remote work | Medical insurance | Mental health support | Mentoring programsExecutive-level Full TimeFrance R2d ago
-
API Design | Backend architecture | Best practices | Command Line | Command-line InterfaceCompany-provided equipment | Continuous learning allowance | Flexible working hours | Fully remote | Health insurance allowanceSenior-level Full TimeFrance R3d ago
-
Alternance Data Analytics & AI (H/F) EUR 41K-55KArtificial Intelligence | Big Data | Data Science | Data analytics | Machine LearningActionnariat salarié | Mutuelle familiale avantageuse | Programme Share for You | Restaurant d'entreprise | Télétravail 2 jours par semaineEntry-level Apprenticeship Part TimeEurope, France, Ile de France, CERGY-PONTOISE R3d ago
-
Analytics Engineer - Pricing data (f/m/d) EUR 48K-48KAWS | Airflow | BigQuery | CI/CD | DBTProvided equipment | Remote work | Technical certifications | Training and developmentEntry-level Full TimeLille, Upline, Nord, France R3d ago
-
Embedded SRE EUR 13KAWS | Ansible | ArgoCD | CI/CD | DockerHealth insurance support | Savings Plan Employer Match | Telework | Training opportunities | Transport supportSenior-level Full TimeParis R3d ago
-
Ingénieur Big Data (F/H) EUR 50K-70KApache Kafka | Apache Spark | Big Data | Data Ingestion | Distributed ComputingFlexible internal work modes | Parental leave support | Telework 3 days per week | Training and ASI AcademyMid-level Full TimeRennes, Brittany, France R3d ago
-
AWS | Autoscaling | Azure | Bash | CI/CDEquity package | Fully remote work within Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeFrance R4d ago
-
CI/CD | CPU | Computer Vision | Data Augmentation | Deep learningHybrid work | Occasional remote work | Professional mentorshipMid-level Full TimeParis, France R4d ago
-
Airbyte | DBT | DBT Cloud | Data Modeling | Data WarehousingRemote work 3 days per week | Structured onboardingExecutive-level Full TimeParis, France R4d ago
-
Airbyte | Basedash | BigQuery | DBT | DBT CloudAutonomy | Fast recruitment process | Flexible remote work | Open office in ParisMid-level Full TimeParis, France R4d ago
-
Azure DevOps | Back-end | Back-end development | Data Modeling | DevOpsCompany events | Telework 2 days per weekMid-level Full TimeParis, France R4d ago
-
Chef de projet GEN AI F/H EUR 35K-40KAI Act | Artificial Intelligence | Budget Management | Change Management | ComplianceEmployee participation plan | Meal allowance | Profit sharing plan | Public transport reimbursement | Remote workEntry-level Full TimeSAINT-MALO, France; C.F.P.R. St Malo R4d ago
-
AI roadmap | Artificial Intelligence | Business case | Business case analysis | Case analysisContinuous learning | Health and wellness programs | TeleworkEntry-level Full TimeIssy-les-Moulineaux, FR R4d ago
-
Ingénieur Databricks Confirmé (F/H) EUR 45K-50KAWS | Apache Spark | CI/CD | DataOps | DatabricksCareer growth | Certification support | Hybrid work | TrainingSenior-level Full TimeBezons, FR R4d ago
-
3NF | Apache Airflow | Azure | Azure Data | Azure Data FactoryAssurance santé | Participation et intéressement | Tickets restaurant | Transport public remboursé | TélétravailSenior-level Full TimeSaint-Priest, France R4d ago
-
QA Automaticien·ne Robot Framework - Nantes F/H EUR 40K-48KAPI Testing | Agile Scrum | Appium | AppiumLibrary | BDDCareer development | Certification support | Meal vouchers | Mutual insurance | Paid time offExecutive-level Full TimeNantes, Pays de la Loire, France R4d ago
-
Applied AI Engineer EUR 50K-80KAgents | Anthropic | BigQuery | ChatGPT | Claude CodeHealth insurance | Meal benefits | Paid time off | RTT | TeleworkEntry-level Full TimeParis R4d ago
-
Agents | Data Modeling | Data Pipelines | Docker | EmbeddingsEquipment support | Flexible working hours | Gifts and rewards | Lunch allowance | No dress codeSenior-level Full TimeFrance R4d ago
-
AWS CloudTrail | AWS CloudWatch | AWS EMR | Apache Beam | Apache SparkCooptation bonuses | Disability support | Employee stock ownership | Equipment allowance | Great Place to Work certificationSenior-level Full TimeLyon, Auvergne-Rhône-Alpes, France R4d ago
-
Applied Gen AI Engineer EUR 70K-80KAWS | Agile | Angular | Azure | CI/CDFlexible working hours | Health insurance | Meal vouchers | Paid time off | TeleworkSenior-level Full TimeLevallois-Perret, IDF, France R4d ago
-
AWS | Amazon Web Services | Apache Hadoop | Apache Kafka | Apache SparkCareer Development Programs | Employee representatives council | Health insurance | Meal vouchers | MentorshipSenior-level Full TimeVilleneuve-d'Ascq, Hauts-de-France, France R4d ago