Data Scientist - AI Evaluation
Tasks
- Build and maintain evaluation datasets benchmarks and scoring frameworks
- Create dashboards and reporting for agent performance
- Define and evolve accuracy metrics across shopping experience
- Design and run experiments for improvements and regressions
- Identify failure modes and edge cases and drive improvements through data
- Partner with ML engineers to validate model changes
- Translate ambiguous product questions into measurable hypotheses
Perks/Benefits
- 401k plan
- Company Offsites and Team Gatherings
- Company holidays
- Equity
- Flexible PTO
- Fully remote work
- Medical, dental & vision coverage
Skills/Tech-stack
A/B | A/B Testing | B testing | Benchmarking | Causal Inference | Dashboards | Data Science | Dataset Engineering | Evaluation metrics | Experimentation | Language Models | Language Processing | Large Language Models | Machine Learning | Natural Language | Natural Language Processing | Ranking | Recommendation Systems
Education
N/A
Roles
Related jobs
-
Principal Data Science - Intelligent Fraud Operations USD 110K-180KArtificial Intelligence | Automation | Case management | Financial Risk Management | Financial risk401k matching | Caregiver leave | Childcare discounts | Dental insurance | Employee assistance programSenior-level Full Time601 S. Tryon Street, NC R6h ago
-
Data Scientist, Product USD 209K-235KA/B | A/B Testing | B testing | Clustering | Data MiningTelecommutingSenior-level Full TimeMenlo Park, CA | Remote, US R8h ago
-
Lead Data Scientist - Healthcare Transformation USD 145K-200KAdvanced Analytics | Agile | CPRS | Clinical informatics | Data Governance401k match | Dental insurance | Federal Holidays | Health insurance | Paid time offSenior-level Full TimeFully Remote - Based in USA R18h ago
-
Applied AI/ML Scientist USD 128K-193KA/B | A/B Testing | AWS | Anthropic | Azure401k match | Annual lifestyle stipend | Child Eldercare Support | Flexible paid time off | Health, dental, vision insuranceMid-level Full TimeRemote US R1d ago
-
Data Scientist USD 172K-203KA/B | A/B Testing | B testing | Data Mining | Data VisualizationTelecommutingEntry-level Full TimeMenlo Park, CA | Remote, US R1d ago
-
Lead Data Scientist USD 97K-230KA/B | A/B Testing | Attribution Modeling | B testing | Causal Inference401 k retirement plan matching | Company recognition program | Education assistance | Flexible work location | Health and wellbeing resourcesSenior-level Full TimeRemote, United States R1d ago
-
Data Scientist - NLP USD 120K-220KAWS | Amazon Textract | Azure Translation Services | BERT | Bedrock401k match | Bonuses | Employer paid health care | Fully remote | Training and development fundsSenior-level Full TimeUnited States - Remote R1d ago
-
Military Advisor- Big Data & Systems Engineer (Part Time/ Remote) (Mission Assurance 4)- 27867 USD 106K-150KAPI Integration | Apache Hadoop | Apache Kafka | Apache Spark | Big Data401k savings plan | Employee assistance programs | Episodic travel | Financial planning tools | Health, dental, and vision plansSenior-level Part TimeCamp HM Smith, HI, Remote, United … R1d ago
-
A/B | A/B Testing | B testing | Causal Inference | Data ModelingEmployer matched 401 k | Exceptional benefits package | Flexible vacation and paid time off | Hybrid work environment | Remote work optionSenior-level Full TimeSanta Monica, CA ; Remote R1d ago
-
API | AWS SageMaker | Agile | Airflow | Amazon Redshift401k with employer match | Adoption, Fertility and Surrogacy Reimbursement | Disability and Critical Illness plans | Emergency backup care | Free access to CEUs and professional developmentSenior-level Full TimeCorp Facilities MPB - 350 Centre … R2d ago
-
Principal, Advanced Analytics – Subscriber Forecasting USD 134K-244KARIMA | Cohort modeling | Data orchestration | Gradient Boosting Machines | Gradient boostingBackground check | Remote workSenior-level Full TimeEl Segundo-CA-2260 E Imperial Hwy, United … R2d ago
-
Data Scientist USD 93K-155KAWS | Clustering | Decision Trees | Ensemble learning | Generalized Linear Models401k | Dental insurance | Medical insurance | PTO | Remote workEntry-level Full TimeDallas TX, United States R2d ago
-
Senior Data Scientist, Data Management USD 110K-138KData Modeling | Data Visualization | Linkage Methods | Machine Learning | Predictive ModelingAnnual leave | Employee assistance program | Flexible work remote | Health insurance | Life assuranceSenior-level Full TimeUS, Blue Bell (ICON), United States R2d ago
-
Lead Data Scientist USD 115K-174KAgile | Data Migration | Data Visualization | ETL | FHIRHealth care plan | Life insurance | Long-term disability | Paid time off | Retirement planSenior-level Full TimeWashington, District of Columbia, United States … R2d ago
-
Data Scientist II - Computer Vision USD 140K-170KComputer Vision | Convolutional Neural Networks | Deep learning | Experiment tracking | Field extractionMid-level Full TimeRemote - US R3d ago
-
Senior Data Scientist - Computer Vision USD 140K-170KComputer Vision | Convolutional Neural Networks | Data Analysis | Deep learning | Fraud DetectionSenior-level Full TimeRemote - US R3d ago
-
Senior Data Scientist USD 170K-205KAWS | Cloud platform | Cybersecurity Risk Scoring | Cybersecurity risk | Data AnalysisHealth benefits | Parental leave | Tuition reimbursement | Unlimited paid time offSenior-level Full TimeAustin (Hybrid) R3d ago
-
Senior Data Scientist USD 170K-205KAWS | Cybersecurity | Data Analysis | Databricks | Exploratory Data AnalysisHealth benefits | Parental leave | Tuition reimbursement | Unlimited PTOSenior-level Full TimeNYC (Hybrid) R3d ago
-
Data Scientist USD 135K-145KAWS | Azure | BI tools | Cause analysis | Clustering401k retirement plan | Employee development stipend | Holiday break | Medical/Dental/Vision insurance | New York office accessEntry-level Full TimeRemote (United States) R5d ago
-
Senior Staff Engineer - Data Scientist USD 150K-190KAmazon Web Services | Anomaly Detection | Apache Airflow | Apache Kafka | Apache SparkContract-to-hire | Remote work | Travel opportunitiesSenior-level Full TimeRemote, REMOTE, United States R5d ago
-
Data Scientist USD 131K-180KAzure Machine Learning | Data Wrangling | Data cleaning | Feature Engineering | Machine LearningRemote workSenior-level Full TimeUnited States - Remote R5d ago
-
Data Scientist L6 - Games Portfolio USD 491K-775KCausal Inference | Data Pipelines | Deep learning | Distributed Systems | Experimentation401k Retirement Plan Employer Match | Disability programs | Family-forming benefits | Flexible spending accounts | Health plansSenior-level Full TimeUSA - Remote, United States R5d ago
-
Postdoctoral Data Analytics Computational Sciences USD 79K-127KAdversarial Networks | Color Transformations | Computer Vision | Convolutional Neural Networks | Deep learning401k | Dental insurance | Floating holidays | Holiday pay | Life insuranceEntry-level Full TimeUS026 PA Spring House - 1400 … R5d ago
-
Data Scientist (Remote) USD 167K-190KDevOps | Git | Machine Learning | Python | R401k match | Charitable Gift Matching | Dental insurance | Employee stock purchase program | Equipment and servicesSenior-level Full TimeRemote - UT, United States R5d ago
-
Senior Data Scientist – Value‑Based Car USD 108K-147KAWS Athena | AWS Redshift | AWS S3 | Agentic AI | Applied Machine LearningRemote within US onlySenior-level Full TimeTX-Remote, United States R5d ago