AI Applied Scientist
Tasks
- Build evaluation dashboards and reporting
- Build scoring frameworks
- Calibrate LLM judges
- Create offline and online evaluation
- Define accuracy metrics
- Design experiments for eval
- Fine tune LLM judges
- Identify failure modes and edge cases
- Improve LLM judge prompting
- Lead applied science evaluation work
- Maintain evaluation datasets
- Mentor science team
- Run automated evaluations before after launches
- Translate questions into measurable hypotheses
- Validate model changes with engineers
Perks/Benefits
- 401k plan
- Company offsites
- Equity
- Flexible PTO
- Medical, dental & vision coverage
- Remote work
- Team gatherings
Skills/Tech-stack
A/B | A/B Testing | Artificial Intelligence | B testing | Benchmarking | Calibration | Causal Inference | Dashboards | Data Analysis | Dataset development | Evaluation science | Experiment design | Fine Tuning | Human Feedback | Information Retrieval | LLM | Language Models | Large Language Models | Learning from Human Feedback | Machine Learning | Machine Learning Experimentation | Prompt engineering | RAG | RLHF | Ranking | Recommendation Systems | Reinforcement Learning | Reinforcement Learning from Human Feedback | Retrieval-Augmented Generation | Statistical Analysis
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Related jobs
-
Machine Learning Scientist USD 263K-322KBatch Prediction | Causal Inference | ETL | Experimentation | ForecastingEquity compensation | Health benefitsMid-level Full TimeSan Mateo, CA, United States R7h ago
-
AI Scientist I USD 129K-207KC plus plus | C# | Deep learning | Hyperparameter Tuning | Image SegmentationDental insurance | Fitness programs | Learning and development | Medical insurance | Mental health supportSenior-level Full TimeSeattle, Washington, United States R12h ago
-
CI/CD | Computer Vision | Data Pipelines | Data Structures | Docker401k plan | FSA | HSA | Medical/Dental/Vision insurance | Paid parental leaveSenior-level Full TimeAustin, Texas or Remote R14h ago
-
Senior Data Scientist USD 149K-181KA/B | A/B Testing | Adjust | Appsflyer | Attribution401k matching | Dental insurance | Flexible paid time off | Health benefits | Parental leaveSenior-level Full TimeAustin, TX R14h ago
-
Bayesian Inference | Causal Inference | Dashboarding | Data Visualization | Experiment designEmployer-matched 401k | Flexible hybrid work environment | Flexible vacation and paid time offMid-level Full TimeSanta Monica, CA/Remote R17h ago
-
Data Scientist - Search USD 175K-234KA/B | A/B Testing | B testing | Experiment design | Information RetrievalHealth plans | OneMedical membershipSenior-level Full TimeUS Remote R17h ago
-
Data Scientist USD 202K-230KAmazon Redshift | Apache Spark | Causal Inference | Experimental Design | Lifetime ValueCareer coaching support | Fitness benefits | Hybrid work schedule | In-office culinary options | Inclusive family building benefitsSenior-level Full TimeSan Francisco R19h ago
-
Staff Data Scientist USD 250K-275KA/B | A/B Testing | Advanced Clustering | Amazon EMR | Amazon S3Hybrid work option | Knowledge sharing sessions | Mentorship opportunities | Telecommuting optionSenior-level Full TimeCA - San Francisco HQ ; … R1d ago
-
Staff Data Scientist USD 250K-275KAWS | Cloud Data | Cloud Data Environments | Credit Risk | Credit risk modelingSenior-level Full TimeCA - San Francisco HQ ; … R1d ago
-
Lead Machine Learning Scientist USD 174K-224KA/B | A/B Testing | Attribution | B testing | Churn modeling401k matching | Company holidays | Dental insurance | Financial wellness support | Flexible PTOSenior-level Full TimeUnited States R1d ago
-
Senior Applied Scientist - Search USD 200KData Science | Fine Tuning | Hybrid search | Information Retrieval | Knowledge graphs401k retirement plan | Growth opportunities | Hybrid work schedule | Medical, dental, and vision insurance | Paid time offSenior-level Full TimeNew York City R2d ago
-
Senior Data Scientist USD 149K-171KAmazon Redshift | Anomaly Detection | Azure | Cloud platform | ClusteringContinuous learning culture | Employee mentorship program | Growth opportunities | Leadership programsSenior-level Full TimeUnited States R2d ago
-
Staff Data Scientist, Safety Insights USD 217K-303KCausal Inference | Difference-in-differences | Experimentation | GenAI | Language Models401k employer match | Family planning support | Flexible vacation | Gender-affirming care | Health care benefitsSenior-level Full TimeRemote - United States R2d ago
-
Sr Data Scientist USD 142K-221KA/B | A/B Testing | Amplitude | B testing | ClassificationFinancial security support | Flexible-hybrid work | Healthcare coverage | Mental health resources | Paid time offSenior-level Full TimeUSA - Illinois - Chicago - … R2d ago
-
Senior Staff Research Scientist, Agentic AI & RL USD 150K-200KDocker | Fine Tuning | LLM Fine-tuning | Language Models | Language ProcessingHigh autonomy | MentorshipSenior-level Full TimeRemote Work( USA), United States R2d ago
-
Director, Data Science & Strategy USD 190K-349KA/B | A/B Testing | Analytics workflows | Artificial Intelligence | B testingExecutive-level Full TimeSan Francisco, United States R2d ago
-
Senior Data Scientist USD 107K-195KAzure | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure Machine LearningSenior-level Full Time6314 Remote/Teleworker US, United States R2d ago
-
Agentic AI | Amazon Bedrock | Amazon Web Services | Azure AI | Cloud ArchitectureSenior-level Full TimeNew York, NY, US, 10001-8604 R2d ago
-
Sr Data Scientist I (Hybrid) USD 95K-158KAWS | Azure | Caret | Data Pipelines | Data Validation401k match | Accidental death insurance | Adoption benefits | Commuter Spending Account | Critical illness insuranceSenior-level Full TimeAlpharetta, GA (Alderman), United States R2d ago
-
Data Scientist, People & Workforce Analytics USD 90K-105KCorrelation Analysis | Data Pipelines | Data cleaning | Feature Engineering | Generative AIHealth benefits package | Paid Holidays | Paid time offMid-level Full TimePA Philadelphia, United States R2d ago
-
Data Scientist - Search USD 175K-234KA/B | A/B Testing | B testing | Information Retrieval | Language ModelsHealth plan coverage | Hybrid work model | OneMedical membershipSenior-level Full TimeUS Remote R2d ago
-
Senior Scientist I Computational Chemistry USD 106K-202KADMET | Artificial Intelligence | Atomistic modeling | C plus plus | Computer Modeling401k | Dental insurance | Health insurance | Paid time off | Vision insuranceSenior-level Full TimeNorth Chicago, IL, United States R2d ago
-
Senior Data Scientist, Acquisition Marketing USD 120K-170KApplied statistics | Automation | Causal Inference | DBT | Data Modeling401k match | Annual performance reviews | Company equipment | Company events | Dental insuranceSenior-level Full TimeAtlanta, GA preferred, Remote R2d ago
-
Data Scientist, Manufacturing Analytics USD 68K-192KAI model | AI model development | Agile | Artificial Intelligence | Data VisualizationAdoption support | Child care assistance | Dental insurance | Employee resource groups | Fertility treatmentsEntry-level Full TimeUnited States R2d ago
-
Sr Applied Scientist, Generative AI/ML USD 142K-270KComputer Vision | Data Curation | Data Generation | Deep learning | Distributed TrainingSenior-level Full TimeSeattle, United States R3d ago