Senior Data Scientist for Copilot Evals
United States, Washington, Redmond
USD 119K-234K Senior-level Full Time
Tasks
- Analyze offline online agreement and regression coverage
- Apply causal and experimental methods
- Automate pipelines for metrics and data quality checks
- Build prioritization framework
- Build scorecards for offline online metrics
- Define taxonomies and metrics
- Design quality gates
- Develop sampling weighting deduplication clustering trend detection
- Establish data provenance privacy responsible use reproducibility governance
- Identify underrepresented gaps
- Measure evaluation coverage
Perks/Benefits
- N/A
Skills/Tech-stack
Anomaly Detection | Causal Inference | Classification | Clustering | Data Governance | Data Quality | Data Visualization | Embeddings | Experimentation | Language Processing | Machine Learning | Metric governance | Natural Language | Natural Language Processing | Privacy-preserving | Privacy-preserving analysis | Python | R | Regression | Reproducibility | SQL | Sampling | Segmentation | Statistical Analysis | Text Analytics
Education
N/A
Roles
Regions
Countries
States
Related jobs
-
A/B | A/B Testing | B testing | Causal Inference | ClusteringMid-level Full TimeMountain View, CA, USA; San Francisco, …6h ago
-
Entry-level Full TimeUnited States, Washington, Redmond18h ago
-
Senior Data Scientist USD 119K-144KAWS | Azure | Classification | Clustering | Data Warehousing401k | Paid sick leave | Paid vacation | Safe TimeSenior-level Full TimeCarmel, Indiana, United States; Eagan, Minnesota, …19h ago
-
Sr. Applied Scientist, AI Evaluation & Quality Systems USD 170K-300KAnomaly Detection | Calibration | Data Annotation | Deep learning | Drift DetectionSenior-level Full TimeSeattle19h ago
-
Lead Data Scientist - Experimentation USD 110K-160KA/B | A/B Testing | AWS | B testing | Causal InferenceEducational Training Programs | Employee assistance program | Health and wellbeing programs | Mentoring | Paid HolidaysSenior-level Full TimeBloomington, IL R20h ago
-
Senior Data and Applied Scientist USD 119K-234KAI Foundry | Azure AI | Azure AI Foundry | Azure Data | Azure Data ExplorerSenior-level Full TimeUnited States, Washington, Redmond20h ago
-
Principal Data Scientist – Supply Chain Operations USD 114K-234KAnalytics Cloud | Cloud Applications | Cloud infrastructure | Dashboards | Data Governance401k matching | Life insurance | Medical/Dental/Vision insurance | Paid Holidays | Paid parental leaveSenior-level Full TimeSanta Clara, CA, United States21h ago
-
Entry-level Full TimeAnnapolis Junction, MD21h ago
-
Senior-level Full TimeAnnapolis Junction, MD22h ago
-
Machine Learning Research Scientist, Evaluations USD 180K-225KBenchmark development | Deep learning | Failure analysis | Fine Tuning | Human FeedbackCommuter stipend | Comprehensive health, dental and vision coverage | Generous PTO | Learning and development stipend | Retirement benefitsEntry-level Full TimeSan Francisco, CA; Seattle, WA; New …22h ago
-
Machine Learning Research Scientist, Post-Training USD 180K-225KDeep learning | Fine Tuning | Human Feedback | Instruction Tuning | Language ModelsCommuter stipend | Health, dental and vision coverage | Learning and development stipend | Paid time off | Retirement benefitsMid-level Full TimeSan Francisco, CA; Seattle, WA; New …22h ago
-
(Senior) Scientist, Computational Neuroscience USD 132K-259KComputational modeling | Data Analysis | Machine Learning | Neural Data | Neural data analysisAnnual incentive program | Healthcare coverage | Retirement benefitsSenior-level Full TimeCambridge, MA USA23h ago
-
Principal, Data/Analytics and Benchmarking USD 129K-186KAI | Benchmarking | Dashboarding | Data Governance | Data ModelingSenior-level Full TimeUnited States R23h ago
-
Lead Data Scientist - Applied AI (P4656) USD 125K-207KAI Agent | AI Agents | AI agent evaluation | Agent Traceability | Agent evaluation401k matching | Dental insurance | Health insurance | Health savings account | Paid time offSenior-level Full TimeCincinnati, OH; Chicago, IL23h ago
-
Staff Data Scientist USD 123K-185KArcGIS | Artificial Intelligence | Automated testing | Bias detection | CI/CD401k | Adoption Assistance | Employee assistance programs | Fitness subsidies | Flexible work arrangementsSenior-level Full TimeAustin,Texas,United States1d ago
-
Data Scientist/Senior Data Scientist USD 136K-183KAgentic Systems | Calculus | Classification | Cloud Native | Distributed ComputingSenior-level Full TimeRedwood City, California, United States1d ago
-
Senior Lead Data Scientist USD 204K-226KCode standards | Data Pipelines | Deep learning | Language Processing | MLOps401k plan | Caregiver referral network | Commuter program | Dental insurance | Disability insuranceSenior-level Full TimeAventura, Florida, United States1d ago
-
Senior Lead Data Scientist USD 204K-226KDeep learning | Language Processing | MLOps | Model Deployment | Model Monitoring401k plan | Commuter program | Dental benefits | Disability insurance | Employee assistance programSenior-level Full TimeBoston, Massachusetts, United States1d ago
-
Senior Lead Data Scientist USD 204K-226KData Manipulation | Deep learning | Language Processing | MLOps | Machine Learning401k plan | Commuter program | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeAustin, Texas, United States1d ago
-
Senior Lead Data Scientist USD 204K-226KDeep learning | Language Processing | MLOps | Machine Learning | Model Monitoring401k plan | Commuter program | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeWashington, District of Columbia, United States1d ago
-
Senior Lead Data Scientist USD 204K-226KBest practices | Clean Code | Data Analysis | Data Manipulation | Deep learning401k plan | Commuter program | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeSeattle, Washington, United States1d ago
-
Senior Data Scientist USD 80KA/B | A/B Testing | B testing | Behavioral Modeling | Business IntelligenceComprehensive benefits | Flexible PTOSenior-level Full TimeUnited States1d ago
-
Principal Applied Scientist for Copilot Evals USD 142K-274KBias | Evaluation | Experimental Design | Experimentation | Information RetrievalSenior-level Full TimeUnited States, Washington, Redmond1d ago
-
Scientist II / Senior Scientist, Computational Biology USD 130K-180KComputational Biology | Data Analysis | Machine Learning | Python | RNA SequencingAnnual incentive program | Healthcare coverage | Retirement benefitsSenior-level Full TimeSomerville, MA1d ago
-
Scientist II / Senior Scientist, Computational Biology USD 130K-180KATAC sequencing | Computational Biology | Data Analysis | Data Science | Machine LearningAnnual incentive program | Healthcare coverage | Retirement benefitsSenior-level Full TimeSomerville, MA USA1d ago