Product Data Scientist — AI Evaluation & Quality
Tasks
- Build online quality dashboards for AI systems
- Convert failures into regression cases
- Extend offline eval suite with datasets judges metrics
- Handle non deterministic evaluation results
- Harden evaluation methodology for judge stability
- Mine failure patterns from production traffic
- Monitor AI resolution rate and customer feedback
- Own eval loop for AI products
- Propose fixes to product and domain experts
- Translate metrics into product decisions
Perks/Benefits
- Continuous professional development
- Hybrid work
- Relocation support
- Remote work
- Stock options
- Work and Swim program
Skills/Tech-stack
Dashboarding | Data Analysis | Databricks | Hypothesis Testing | LLM | Machine Learning | Python | Quality analytics | RAG | SQL | Sampling | Statistics | Variance
Education
N/A
Roles
Related jobs
-
Senior Data Scientist UAH 1377K-1549KAWS | Azure | BigQuery | ClickHouse | Data PipelinesConference access | Equipment budget | Equity options | Extra days off | Flexible work scheduleSenior-level Full TimeBulgaria; Croatia; Estonia; Greece; Hungary; Latvia; … R11d ago
-
Lead Data Scientist EUR 62K-73KAgile | Azure | Classification | Data cleaning | DockerContinuous learning budget | Hobby grant | Mental health support | Private healthcare | Share optionsSenior-level Full TimeFinland; Latvia; Remote; Spain; Tallinn, Harju, … R20d ago
-
BERT | Community Segmentation | Generative AI | LLM Frameworks | Language ModelsRemote workSenior-level Full TimeTallinn, Estonia R1mo ago