AI Applied Scientist
Tasks
- Build evaluation dashboards and reporting
- Build scoring frameworks
- Calibrate LLM judges
- Create offline and online evaluation
- Define accuracy metrics
- Design experiments for eval
- Fine tune LLM judges
- Identify failure modes and edge cases
- Improve LLM judge prompting
- Lead applied science evaluation work
- Maintain evaluation datasets
- Mentor science team
- Run automated evaluations before after launches
- Translate questions into measurable hypotheses
- Validate model changes with engineers
Perks/Benefits
- 401k plan
- Company offsites
- Equity
- Flexible PTO
- Medical, dental & vision coverage
- Remote work
- Team gatherings
Skills/Tech-stack
A/B | A/B Testing | Artificial Intelligence | B testing | Benchmarking | Calibration | Causal Inference | Dashboards | Data Analysis | Dataset development | Evaluation science | Experiment design | Fine Tuning | Human Feedback | Information Retrieval | LLM | Language Models | Large Language Models | Learning from Human Feedback | Machine Learning | Machine Learning Experimentation | Prompt engineering | RAG | RLHF | Ranking | Recommendation Systems | Reinforcement Learning | Reinforcement Learning from Human Feedback | Retrieval-Augmented Generation | Statistical Analysis
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R6d ago
-
Senior Data Scientist, Rider New Products USD 148K-185KCausal Inference | Counterfactual analysis | Experimental Design | Incrementality Measurement | Machine Learning401k plan | Child care benefits | Dental insurance | Discretionary paid time off | Family building benefitsSenior-level Full TimeSan Francisco, CA R21h ago
-
Senior Data Scientist, Rider New Products USD 148K-185KCausal Inference | Causal Models | Counterfactual analysis | Experiment design | Incrementality Measurement401k | Child care benefits | Dental insurance | Family building benefits | Health insuranceSenior-level Full TimeNew York, NY R21h ago
-
Data Scientist II USD 107K-124KA/B | A/B Testing | Amplitude | Apache Spark | B testing401k match | Company winter break | Dental coverage | Disability coverage | Free snacks and drinksMid-level Full TimeCrystal City R22h ago
-
Lead Data Scientist, Experimentation Platform USD 240K-260KCausal Inference | Data Processing | Holdout Experiments | Hypothesis Testing | Incremental LiftHybrid work model | Onsite Days Per WeekSenior-level Full TimeStrava SF R23h ago
-
Staff Applied Scientist (Distribution Center) USD 191K-287KApproximate Dynamic Programming | Chain management | Decision analysis | Demand forecasting | Dynamic Programming401k match | Dental insurance | Health insurance | Home office stipend | Mental health supportSenior-level Full TimeRemote - US R1d ago
-
Director I, Data Science, Enterprise Data & Data Science USD 125K-201KData Modeling | Generative AI | Knowledge graphs | Language Processing | MLOpsProfessional development | Workplace flexibilityExecutive-level Full TimeBoston, Massachusetts, United States R1d ago
-
Staff Data Scientist, Marketing USD 120K-150KAd Ranking | Ad Ranking Algorithms | Amazon Web Services | Collaborative Filtering | Content-Based FilteringSenior-level Full TimeUnited States (remote) R1d ago
-
Staff Applied Scientist, AdTech USD 120K-150KAWS | Ad Ranking | Collaborative Filtering | Content-Based Recommendation | Content-basedSenior-level Full TimeUnited States (remote) R1d ago
-
Lead Applied Scientist, Marketing USD 120K-150KAWS | Ad Ranking | Ad Ranking Algorithms | Collaborative Filtering | Content-Based FilteringSenior-level Full TimeUnited States (remote) R1d ago
-
Lead Data Scientist, AdTech USD 120K-150KAWS | Ad Ranking | Ad Ranking Algorithms | Collaborative Filtering | Content-Based FilteringDiverse, inclusive culture | Profit-sharing bonus | Remote-first cultureSenior-level Full TimeUnited States (remote) R1d ago
-
Senior Applied Scientist - Search USD 200K-200KData analytics | Fine Tuning | Hybrid search | Information Retrieval | Knowledge graphs401k retirement | Annual leave | Growth opportunities | Health insurance | Hybrid scheduleSenior-level Full TimeNew York City R1d ago
-
Machine Learning Scientist, RL & Autonomous Discovery USD 132K-200KBayesian optimization | Black-box | Black-box optimization | Distributed Training | InferenceDental insurance | Disability coverage | Health insurance | Life insurance | Retirement planMid-level Full TimeRemote (United States) R1d ago
-
Causal Inference | Drug Discovery | Generative Models | Inference | MLOpsDental insurance | Disability coverage | Health insurance | Life insurance | Retirement safe harbor contributionMid-level Full TimeRemote (United States) R1d ago
-
Lead, Data Science Analyst USD 110K-150KAlgorithms | Boosted Decision Trees | Decision Trees | Deep learning | Language Processing401-k match | Employee stock purchase plan | Health insurance | Hybrid work schedule | Paid HolidaysSenior-level Full TimeHQ Wilmington DE Management Office, United … R1d ago
-
Senior DS/ML Scientist - Search Science USD 115K-170KDeep learning | Dense retrieval | Feature Engineering | Generative Retrieval | Information Retrieval401k plan | Annual bonus | Dental insurance | Health insurance | Paid HolidaysSenior-level Full TimeUS Remote, United States R1d ago
-
Machine Learning Staff Scientist at NSF-NCEMS USD 65K-122KAutoregressive models | CNN | Causal Inference | Classification | ClusteringSenior-level Full TimePenn State University Park, United States R1d ago
-
Data Scientist USD 90K-120KAWS | Agile | Azure | CI/CD | Cloud PlatformsFlexible Fridays Remote Work | Hybrid work schedule | Relocation providedMid-level Full TimeUSA - General Office, United States R1d ago
-
Principal Applied Scientist, Agentforce Operations USD 197K-344KAirflow | Automatic Differentiation | Data Versioning | Deep Learning Model Training | Deep learning401k | Dental insurance | Employee stock purchasing program | Health insurance | Life and disability insuranceSenior-level Full TimeWashington - Seattle Metro - Remote, … R1d ago
-
Senior Data Scientist USD 132K-195KAgile | Apache Spark | Bias Mitigation | Data Wrangling | Data provenanceRemote work flexibilitySenior-level Full TimeRemote, OR, United States R1d ago
-
Data Scientist USD 125K-195KAWS SageMaker | Agile | Azure Machine Learning | Backlog Grooming | Data AnalysisFull remote flexibilitySenior-level Full TimeRemote, OR, United States R1d ago
-
Data Scientist (AHL) USD 140K-160KAgile | BI Dashboards | Credit Risk | Credit risk modeling | Data analytics401k employer match | Achieve Care Fund | Employee assistance program | Employee resource groups | FSA optionSenior-level Full TimeSan Mateo, CA, United States R1d ago
-
Data Scientist (AHL) USD 140K-160KBI Dashboards | Credit Risk | Credit risk modeling | Descriptive Analytics | GCP401k match | Dental insurance | Employee assistance program | FSA | HSASenior-level Full TimeTempe, AZ, United States R1d ago
-
Associate Data Scientist USD 48K-56KAPI Development | Big Data | Data Visualization | Deep learning | DockerFlexible work options | Health coverage | Paid time off | Parental leave | Professional developmentMid-level Full TimeHamilton, Township, New Jersey R1d ago
-
Analytics Lead USD 160K-240KA/B | A/B Testing | Agent architecture | Airtable | B testing401k | Dental insurance | Medical insurance | Vision insuranceSenior-level Full TimeRedwood City, CA (Hybrid) R1d ago