Product Data Scientist — AI Evaluation & Quality
Tasks
- Build online quality dashboards for AI systems
- Convert failures into regression cases
- Extend offline eval suite with datasets judges metrics
- Handle non deterministic evaluation results
- Harden evaluation methodology for judge stability
- Mine failure patterns from production traffic
- Monitor AI resolution rate and customer feedback
- Own eval loop for AI products
- Propose fixes to product and domain experts
- Translate metrics into product decisions
Perks/Benefits
- Continuous professional development
- Hybrid work
- Relocation support
- Remote work
- Stock options
- Work and Swim program
Skills/Tech-stack
Dashboarding | Data Analysis | Databricks | Hypothesis Testing | LLM | Machine Learning | Python | Quality analytics | RAG | SQL | Sampling | Statistics | Variance
Education
N/A
Roles
Related jobs
-
Senior Data Scientist EUR 60K-85KAnomaly Detection | CatBoost | Computer Vision | Deep learning | Fraud DetectionExtra recharge days | Health and sports budget | Learning and development budget | Medical/Dental/Vision insurance | Relocation supportSenior-level Full TimeTallinn, Spain (Remote) R26d ago
-
BERT | Community Segmentation | Generative AI | LLM Frameworks | Language ModelsRemote workSenior-level Full TimeTallinn, Estonia R29d ago
-
Data Scientist EUR 45K-68KClassification | Clickstream Data | Cloud Platforms | Clustering | Feature EngineeringMid-level Full TimeTallinn, Remote, EE R29d ago