Product Data Scientist — AI Evaluation & Quality
Tasks
- Build online quality dashboards for AI systems
- Convert failures into regression cases
- Extend offline eval suite with datasets judges metrics
- Handle non deterministic evaluation results
- Harden evaluation methodology for judge stability
- Mine failure patterns from production traffic
- Monitor AI resolution rate and customer feedback
- Own eval loop for AI products
- Propose fixes to product and domain experts
- Translate metrics into product decisions
Perks/Benefits
- Continuous professional development
- Hybrid work
- Relocation support
- Remote work
- Stock options
- Work and Swim program
Skills/Tech-stack
Dashboarding | Data Analysis | Databricks | Hypothesis Testing | LLM | Machine Learning | Python | Quality analytics | RAG | SQL | Sampling | Statistics | Variance
Education
N/A
Roles
Related jobs
-
Staff Machine Learning Scientist EUR 58K-86KAWS | Datadog | Deep learning | Generative AI | Information RetrievalFlexible working hours | Hybrid work model | Remote work flexibility | Work-life balanceSenior-level Full TimeEstonia - Tallinn - Office R21h ago
-
Senior Data Scientist UAH 1377K-1549KAWS | Azure | BigQuery | ClickHouse | Data PipelinesConference access | Equipment budget | Equity options | Extra days off | Flexible work scheduleSenior-level Full TimeBulgaria; Croatia; Estonia; Greece; Hungary; Latvia; … R1mo ago
-
Lead Data Scientist EUR 62K-73KAgile | Azure | Classification | Data cleaning | DockerContinuous learning budget | Hobby grant | Mental health support | Private healthcare | Share optionsSenior-level Full TimeFinland; Latvia; Remote; Spain; Tallinn, Harju, … R1mo ago