Senior Data Scientist, AI Scoring & Evaluation
Tasks
- Automate workflows for product and engineering partners
- Build calibration methods
- Conduct drift detection
- Design evaluator and rubric anchoring
- Implement scoring measurement improvements
- Investigate assessment decisions with analytics studies
- Own scoring quality end to end
- Perform bias diagnostics
- Produce audit ready evidence for AI governance and InfoSec
- Run continuous evaluation harness
- Run root cause analysis for incorrect scores
- Set and track assessment quality KPIs
- Validate scoring changes through pre release gate
Perks/Benefits
- N/A
Skills/Tech-stack
AI Act | Bias detection | Calibration | Classification accuracy | Data Visualization | Data analytics | Drift Detection | EU AI | EU AI Act | Experiment design | GDPR | LLM Output Evaluation | Language Models | Language Processing | Large Language Models | Machine Learning | Model Evaluation | Natural Language | Natural Language Processing | Output Evaluation | Prompt engineering | Python | Reliability analysis | SQL | Statistical Analysis
Education
N/A
Roles
Related jobs
-
Data Scientist USD 128K-214KData Mining | Data Modeling | Data Visualization | Data reporting | Language ProcessingHealth insurance | Holiday pay | Learning and development | Life insurance | Long-term disabilityMid-level Full TimeUSA-MD-Patuxent River1d ago
-
Commercial Data Scientist / Data Analyst - FTC GBP 45K-54KA/B | A/B Testing | B testing | Bayesian Inference | Demand forecastingAnnual credit for discount on easyJet holidays | Discounted staff travel scheme for friends and family | Electric vehicle lease salary sacrifice scheme | Flexible benefits package | Hybrid working 2 days a weekMid-level Full TimeUnited Kingdom-London-London Luton Airport R1d ago
-
Commercial Data Scientist / Data Analyst GBP 43K-54KBayesian Inference | Data Pipelines | Data Quality | Demand forecasting | Dynamic PricingAnnual credit for holiday discounts | Discounted staff travel | Electric vehicle lease scheme | Flexible benefits | Hybrid workMid-level Full TimeUnited Kingdom-London-London Luton Airport R1d ago
-
A/B | A/B Testing | B testing | C++ | Canary ReleaseNone Full TimeSeattle, Washington, United States1d ago
-
Research Scientist, Machine Learning USD 117K-173KC++ | Data Analysis | Deep learning | Dimensional data | EvaluationMid-level Full TimeSunnyvale, CA1d ago
-
Computer Vision | Data Curation | Generative Models | Language Models | Large-scaleMid-level Full TimeMenlo Park, CA | Seattle, WA …1d ago
-
Classification | Clustering | Data Analysis | Data Quality | Data profilingEntry-level Full TimeAmsterdam, NH, NL1d ago
-
Applied Data Scientist and Visualization Specialist - Hardware Engineering Product Design USD 143K-256K3D Data Analysis | 3D data | Anomaly Detection | Computational Geometry | Computer GraphicsMid-level Full TimeSunnyvale1d ago
-
Data Analyst II/Data Scientist II USD 95K-135KAnomaly Detection | Artificial Intelligence | Data Analysis | Data Lifecycle Management | Data analyticsEntry-level Full TimeRosslyn, VA, US R1d ago
-
Senior Data Scientist INR 400K-600KAgentic Frameworks | Amazon SageMaker | Bagging | Boosting | Cloud ComputingBirthday leave | Employee assistance programme | Flexible benefits | Medical insurance | Parental leaveEntry-level Full TimeBangalore, India1d ago
-
AI Research Scientist - GenAI USD 165K-185KC plus plus | Deep learning | GenAI | Inference Optimization | Language ModelsMid-level Full TimePittsburgh, PA, United States1d ago
-
AI Research Scientist II USD 139K-223KAction Recognition | Anomaly Detection | Computer Vision | Data labeling | Dataset curationDental insurance | Employee resource groups | Fitness programs | Learning and development programs | Medical insuranceEntry-level Full TimeSeattle, Washington, United States R1d ago
-
Associate, Client Intelligence (Data Scientist) USD 98K-132KAWS | Apache Spark | Azure | Big Data | Cloud platformEmployee resource groups | Matching gift program | Professional development | Sustainability and Climate Change Initiatives | Volunteerism programMid-level Full TimeNew York, NY, US, 100011d ago
-
VP / AVP, GenAI Engineer/Scientist, Innovation Group SGD 130K-200KAgent evaluation | Audit Logging | Bias Testing | Classification | Data PreparationExecutive-level Full TimeCentral Region (City Area), Singapore1d ago
-
Agentic AI | Audit Logging | Bias Testing | Calibration | Cause analysisExecutive-level Full TimeCentral Region (City Area), Singapore1d ago
-
VP, Data Scientist, Enterprise AI, Innovation Group SGD 130K-261KAgentic AI | Agile project management | Boosting | Data Visualization | Deep learningExecutive-level Full TimeCentral Region (City Area), Singapore1d ago
-
Agile project management | Bayesian Inference | Causal Inference | Data Visualization | Deep learningExecutive-level Full TimeCentral Region (City Area), Singapore1d ago
-
Cientista de Dados Sênior BRL 18K-18KAWS SageMaker | Agile | Backtesting | Credit Risk | Data AnalysisDay off | Dental insurance | Extended parental leave | Gym membership | Health insuranceSenior-level Full TimeBrasil R1d ago
-
Data Teaching Expert USD 14K-30KData Analysis | Data Visualization | Data Wrangling | Excel | MatplotlibCareer development | Digital office tools | International team | Professional feedback | Remote workSenior-level Full TimeTegucigalpa, Honduras, HN R1d ago
-
Data Teaching Expert MXN 480K-550KData Visualization | Data Wrangling | Excel | Matplotlib | NumPyCollaborative learning impact | Digital office tools | International experience | Professional feedback | Remote workSenior-level Full TimeMexico City, Mexico City, MX R1d ago
-
Lead Applied AI and Data Scientist | IFS Copperleaf CAD 113K-152KA/B | A/B Testing | Anomaly Detection | B testing | Causal InferenceSenior-level Full TimeVancouver, British Columbia, Canada R1d ago
-
Entry-level Full TimeBrasil R1d ago
-
Data Scientist Staff USD 100K-134KCassandra | Data Modeling | Data Pipelines | Data scrubbing | Hadoop401k company match | Adoption Assistance | Appliance purchase plan | Commuter benefits | Electric vehicle incentive programSenior-level Full TimeFt Worth, Texas, United States1d ago
-
Applied Scientist, AI Economics (TokenOps, FinOps) USD 101K-193KCVaR | Causal Inference | Chance constraints | Conformal Prediction | Data LineageMid-level Full TimeUnited States1d ago
-
Associate Scientist, Computational Biology USD 93K-112KAPIs | AWS | Airflow | Azure | BioinformaticsCompetitive benefits | PTO | Stock Option PlansMid-level Full TimePhiladelphia, PA R1d ago