Product Data Scientist — AI Evaluation & Quality (Remote)
Tasks
- Build online quality dashboards
- Convert failure patterns into regression cases
- Handle non determinism
- Harden judge stability
- Mine failure patterns from real traffic
- Own and extend offline evaluation suite
- Propose fixes to product and domain experts
- Translate evaluation metrics into product decisions
Perks/Benefits
- Hybrid work
- Personal development opportunities
- Relocation support
- Remote work
- Stock options
- Work-life balance program
Skills/Tech-stack
Claude Code | Databricks | DeepEval | Hypothesis Testing | LLM-as-a-Judge | Python | SQL | Sampling | Statistics | Variance
Education
N/A
Roles
Related jobs
-
Applied ML Scientist EUR 80K-108KA/B | A/B Testing | Airflow | B testing | Batch ProcessingHybrid work model | Remote work flexibilityMid-level Full TimePortugal - Lisbon - Hybrid R1mo ago
-
Senior Data Scientist and AI Dev EUR 55K-78KAI Services | AWS | Azure AI | Azure AI Services | Azure Machine LearningDiscounts | Full-time contract | Health insurance | Meal allowance | Paid vacationSenior-level Full TimeLisbon, Portugal R1mo ago