Product Data Scientist — AI Evaluation & Quality (Remote)
Tasks
- Build online quality dashboards
- Convert failure patterns into regression cases
- Handle non determinism
- Harden judge stability
- Mine failure patterns from real traffic
- Own and extend offline evaluation suite
- Propose fixes to product and domain experts
- Translate evaluation metrics into product decisions
Perks/Benefits
- Hybrid work
- Personal development opportunities
- Relocation support
- Remote work
- Stock options
- Work-life balance program
Skills/Tech-stack
Claude Code | Databricks | DeepEval | Hypothesis Testing | LLM-as-a-Judge | Python | SQL | Sampling | Statistics | Variance
Education
N/A
Roles
Related jobs
-
APIs | AWS Lambda | AWS SageMaker | Amazon EC2 | Amazon S3Flexible autonomy | Fully remote | Knowledge sharing | Mentorship | Short term B2B contractSenior-level Contract Full TimePortugal R4d ago
-
Applied ML Scientist EUR 80K-108KA/B | A/B Testing | Airflow | B testing | Batch ProcessingHybrid work model | Remote work flexibilityMid-level Full TimePortugal - Lisbon - Hybrid R12d ago
-
Data Scientist EUR 45K-68KAd-Tech | Classification | Clickstream analysis | Clustering | Digital marketingMid-level Full TimeLisbon, Remote, PT R1mo ago
-
Senior Data Scientist and AI Dev EUR 55K-78KAI Services | AWS | Azure AI | Azure AI Services | Azure Machine LearningDiscounts | Full-time contract | Health insurance | Meal allowance | Paid vacationSenior-level Full TimeLisbon, Portugal R1mo ago
-
Experience Data Scientist EUR 45K-70KAWS | Azure | Causal Inference | CausalImpact | Data AnalysisAnnual team trip | Conference budget | Flexible working hours | Health insurance | Laptop and peripheralsSenior-level Full TimePortugal - Remote R1mo ago