AI QA Trainer - LLM Evaluation
Tasks
- Analyze model failures and identify patterns
- Assess hallucination detection and factual accuracy
- Automate testing with Python and SQL
- Collaborate on evaluation metrics and quality monitoring
- Conduct adversarial testing and red teaming
- Design evaluation frameworks
- Develop evaluation rubrics and benchmarks
- Document findings and root cause analysis
- Evaluate large language models
- Execute test plans and regression suites
- Provide recommendations for prompts and guardrails
- Validate tool usage and workflow reliability
- Verify grounding and retrieval augmented generation outputs
Perks/Benefits
- Flexible contract opportunity
- Fully remote work
- Health insurance not included
- Opportunity to work on advanced AI systems
- Paid time off not included
Skills/Tech-stack
Adversarial Testing | Bias Assessment | Data Science | Experiment tracking | LLM Evaluation | Language Models | Large Language Models | Machine Learning | Prompt engineering | Python | Red Teaming | Regression testing | Retrieval-Augmented Generation | SQL | Safety testing | Test automation
Education
Roles
AI | AI QA | AI QA Engineer | Engineer | Learning Engineer | Machine Learning Engineer | QA Engineer
Related jobs
-
Cloud Computing | Data Modeling | Docker | GPU Computing | KubernetesCollaborative & Inclusive Culture | Four-day work week | Fully remote | Open source contribution opportunities | Unlimited paid time offSenior-level Full TimeSpain R22h ago
-
Senior-level Full TimeMálaga, AN, Spain R23h ago
-
Especialista en GEN AI (100% Remoto) EUR 39K-48KAI Tooling | Agent architecture | Amazon SageMaker | Cloud Integration | Generative AI100% remote | Childcare support | Expleo Academy | Language learning opportunities | Meal vouchersSenior-level Full TimeBarcelona, CT, Spain R1d ago
-
ChromaDB | Databricks | Elasticsearch | Embeddings | FaissEmployee assistance program | Flexible working environment | LinkedIn Learning | Volunteer time offSenior-level Full TimeMadrid, M, Spain R1d ago
-
Senior Data Engineer (Python/Scala) EUR 54K-74KAgile | Amazon EMR | Amazon Kinesis | Amazon Web Services | Apache AirflowContinuous learning | Hybrid work model | Knowledge sharingSenior-level Full TimeMadrid, Madrid, Spain (Hybrid) R1d ago
-
AWS | Agent systems | Artificial Intelligence | Azure | Batch Machine LearningContract opportunity | Flexible consulting environment | Fully remote | Potential extension | US time zone overlapSenior-level Contract Full TimeSpain R1d ago
-
Cloud Data | Cloud Data Warehouse | Data Lineage | Data Modeling | Data PipelinesFlexible working arrangements | Fully remote Europe | High autonomy | Professional growth opportunitiesSenior-level Full TimeSpain R1d ago
-
Apache Spark | Application Insights | Asset bundles | Azure Data | Azure Data FactorySenior-level Full TimeValència, Valencian Community, Spain - Remote R2d ago
-
Senior AI Engineer (m/f/d) USD 100K-140K.NET | AI Search | AIOps | API Integration | AWS100 percent salary during sick leave | 4 weeks remote work per year | Childcare support | Coursera access | Experience daysSenior-level Full TimeMadrid, MD, Spain R2d ago
-
A/B | A/B Testing | B testing | Distributed Systems | Evaluation methodologyAnnual leave | Global culture | Remote work | Restricted stock unitsSenior-level Full TimeSpain (Remote) R2d ago
-
Agent systems | Agentic AI | Data Pipelines | Experimentation | LLM EvaluationHybrid workMid-level Full TimeMadrid - Hybrid, Barcelona - Hybrid R2d ago
-
Senior AI Engineer (m/f/d) EUR 36K-45K.NET | AI Search | AIOps | API Integration | AngularChildcare support | Coursera access | Experience days | Flexible hours | Free specialist access for health wellbeingSenior-level Full TimeGranada, AN, Spain R2d ago
-
AI Agents | AI Evaluation | AI FinOps | AI Observability | AI systemsCorporate events access | Fully remote work | Medical insurance | Mental health support | Monthly benefits budgetExecutive-level Full TimeSpain R2d ago
-
API Design | Backend architecture | Command Line | Command-line Interface | Database systemsCompany-provided equipment | Continuous learning allowance | Flexible working hours | Fully remote work | Health insurance allowanceSenior-level Full TimeSpain R3d ago
-
Senior Data Engineer Databricks (m/f/d) EUR 36K-45KAmazon Web Services | Apache Spark | Azure | CI/CD | Cloud platform100 percent salary during sick leave | Childcare support | Coursera access | Flexible compensation | Flexible scheduleSenior-level Full TimeReus, CT, Spain R3d ago
-
Lead, Finance Analytics & Enablement AI/ML EUR 56K-77KAmplitude | Automation | Business Intelligence | Cloud platform | Data EngineeringAccident insurance | Co-working access | Gym allowance | Life insurance | Parental leaveSenior-level Full TimeSpain - Remote R3d ago
-
Automated testing | CI/CD | Computer Vision | Conda | DockerBirthday off | Home-office allowance | Hybrid work | Local holidays | Paid vacationSenior-level Full TimeMadrid, Remote EURO, Remote US R3d ago
-
A/B | A/B Testing | AWS | Autoscaling | B testingEquity package | Fully remote work Europe | Healthcare coverage | Paid parental leave | Professional growthMid-level Full TimeSpain R4d ago
-
Data Pipelines | Data Visualization | Databricks | Forecasting | LLMFlexible schedule | Hybrid work | Professional development | Remote work flexibilitySenior-level Full TimeMadrid - Castellana, Spain R4d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R4d ago
-
Machine Learning Engineer EUR 78K-90KAWS | Amazon SageMaker | Artifact management | Azure | CI/CDEmployee stock-option | Flexible paid time off | Hybrid work options | Paid family leave | Remote work up to 30 daysMid-level Full TimeMadrid, Spain R4d ago
-
Analytics Engineer USD 100KAmazon Redshift | Apache Airflow | CI/CD | DBT | Data LineageEnglish as office language | In person work 3 days per week | Relocation supportMid-level Full TimeBarcelona R4d ago
-
Alerting | Amazon Redshift | Aurora PostgreSQL | Batch Processing | CI/CDCollaboration with cross-functional teams | Fully remote | Opportunity for continuous improvement | Ownership over data reliability strategySenior-level Full TimeSpain R4d ago
-
Senior Data Engineer EUR 70K-90KCI/CD | ClickHouse | Docker | Elasticsearch | KafkaFlexible working hours | Learning time | Location Based Social Benefits | Remote-first | Team eventsSenior-level Full TimeGermany /Spain / Romania (Remote) R5d ago
-
Staff/Lead Python Engineer (FastAPI, Orchestration) EUR 82K-112KAsynchronous programming | Cloud platform | Concurrent programming | FastAPI | GenAIAnnual learning and development budget | Co-working budget | Company laptop | Enhanced parental leave | Flexible scheduleSenior-level Full TimeRemote Anywhere R6d ago