Product Data Scientist — AI Evaluation & Quality (Remote)
Tasks
- Build offline eval suite
- Convert failures into regression cases
- Create online quality dashboards
- Define datasets judges metrics
- Harden judge stability and handle non determinism
- Mine production failures from traffic
- Monitor resolution rate CSAT feedback
- Own eval loop for AI products
- Propose product fixes with domain experts
- Translate metrics into decisions
Perks/Benefits
- Continuous professional development
- Hybrid work
- Relocation support
- Remote work
- Stock options
- Work and Swim program
- Work in EU
Skills/Tech-stack
Claude Code | Data Visualization | Databricks | DeepEval | Hypothesis Testing | LLM-as-judge | Language Processing | Machine Learning | Natural Language | Natural Language Processing | Python | RAG | Retrieval-Augmented Generation | SQL | Sampling | Statistics | Variance analysis
Education
N/A
Roles
Related jobs
-
Senior Data Scientist UAH 1377K-1549KAWS | Azure | BigQuery | ClickHouse | Data PipelinesConference access | Equipment budget | Equity options | Extra days off | Flexible work scheduleSenior-level Full TimeBulgaria; Croatia; Estonia; Greece; Hungary; Latvia; … R14d ago
-
Data Scientist EUR 52K-67KData Analysis | Exploratory Data Analysis | Feature Engineering | Git | GitHubHybrid work model | Private insurance | Remote work | Top-tier tech gear | Training budgetMid-level Full TimeAthens, Attica, Greece R1mo ago
-
Senior Data Scientist (Marketing Mix Models) EUR 54K-72KAWS | Azure | Data Visualization | Demand forecasting | EconometricsHybrid work model | Private insurance | Remote work option | Top-tier tech gear | Training budgetSenior-level Full TimeAthens, Attica, Greece R1mo ago