Senior Applied Scientist, Multilingual AI Evaluation
Tasks
- Build multilingual LLM evaluation tooling
- Collaborate with ML researchers and measurement scientists
- Create human evaluation protocols
- Design benchmarks and datasets
- Design evaluation methodology
- Evaluate AI bias and fairness across languages
- Implement evaluation methods in Python
- Validate evaluation methodology across languages
Perks/Benefits
- N/A
Skills/Tech-stack
Benchmark design | Bias and fairness | Bias and fairness evaluation | Cross-lingual modeling | Dataset design | Fairness evaluation | Fine Tuning | Human evaluation | Internationalization | JAX | LLM Evaluation | Language Models | Large Language Models | Localization | PyTorch | Python | Reproducibility | Rubric Design | Sociolinguistics | Statistical rigor
Education
Related jobs
-
Product Data Scientist, Android USD 138K-197KData Modeling | Data Validation | Key Performance Indicators | Performance indicators | PythonMid-level Full TimeMountain View, CA, USA2h ago
-
Data Quality | Data collection | Data-collection pipelines | Experiment design | Feature ExperimentationSenior-level Full TimeNew York, NY, USA2h ago
-
Senior Research Data Scientist, Google Display Ads USD 174K-252KDatabase querying | Machine Learning | Mathematical Modeling | Python | RSenior-level Full TimeMountain View, CA, USA2h ago
-
Senior Data Scientist USD 169K-190KA/B | A/B Testing | B testing | Data Analysis | Data PipelinesCommuter benefits | Disability benefits | Health insurance | Life insurance | Paid time offSenior-level Full TimeSanta Monica, California9h ago
-
Senior Research Data Scientist USD 330K-375KCausal Inference | Difference-in-differences | Distributed Computing | Double Machine Learning | Doubly RobustDisability benefits | Equity awards | Health insurance | Life insurance | Paid time offSenior-level Full TimeNew York, New York9h ago
-
Research Engineer, Knowledge Graph Intelligence USD 175K-250KAttention Mechanisms | Data Augmentation | Data Preprocessing | Deep learning | Embeddings401k match | Employee affinity groups | Family leave | Health care benefits | Parental leaveMid-level Full TimeNew York, NY13h ago
-
Senior Applied Scientist, Reinforcement Learning USD 189K-250KAgent automation | Human Feedback | LLM Training | LLM-as-judge | Language ProcessingSenior-level Full TimeMenlo Park, CA13h ago
-
Staff Applied Scientist, Reinforcement Learning USD 190K-270KLLM Training | LLM-as-judge | Language Models | Language Processing | Large Language ModelsSenior-level Full TimeMenlo Park, CA13h ago
-
Sr. Data Scientist USD 226K-287KA/B | A/B Testing | Attribution windows | B testing | Causal InferenceRemote work | TelecommutingSenior-level Full TimeNew York, NY, US14h ago
-
AWS Bedrock | AWS Bedrock AgentCore | AWS Cloud | Agent systems | Bedrock AgentCoreSenior-level Full TimeNew York, NY, United States15h ago
-
Data Curator USD 70K-88KAutomation | Business Analytics | Data Processing | Data Quality | Data ValidationMid-level Full TimeHybrid, US R17h ago
-
Staff Data Scientist USD 170K-225KA/B | A/B Testing | B testing | Causal Inference | DBT401k match | Discounts | Education stipend | Employer-paid health insurance | Flexible time offSenior-level Full TimeSan Francisco, California, United States17h ago
-
Senior Data Scientist USD 80KA/B | A/B Testing | B testing | Causal Inference | ClassificationFlexible PTO | Health insurance | Paid time off | Retirement benefitsSenior-level Full TimeUnited States18h ago
-
AI/ML Data Scientist USD 176K-222KCloud Computing | Data analytics | Databricks | Deep learning | Language ProcessingMid-level Full TimeSunnyvale, CA, United States18h ago
-
Senior Bioinformatics Scientist USD 150K-189KAWS | Analytical validation | Assay Development | Benchmarking | BioinformaticsSenior-level Full TimeRemote ; San Diego, California, United … R18h ago
-
Sr. Bioinformatics Scientist USD 150K-189KAWS | AWS Batch | AWS Lambda | Analytical validation | BioinformaticsSenior-level Full TimeRemote ; South San Francisco, California, … R18h ago
-
Sr. Data Scientist - Forecasting USD 167K-227KAPIs | Data Pipelines | Hypothesis Testing | JSON | Machine LearningSenior-level Full TimeUnited States18h ago
-
Senior Data Scientist - Fraud (Hybrid) USD 96K-125KAI | Data Analysis | Feature Engineering | Fraud Detection | Machine Learning401k matching | Dental insurance | Health insurance | Hybrid schedule | Mental health benefitsSenior-level Full TimeChicago, IL R19h ago
-
Data Scientist - Pricing & Profitability (Hybrid) USD 87K-110KA/B | A/B Testing | B testing | Machine Learning | Python401k matching | Dental insurance | Health insurance | Hybrid schedule | Mental health benefitsMid-level Full TimeChicago, IL R19h ago
-
Principal Data Scientist USD 142K-304KCloud Computing | Data Analysis | Data Ethics | Data Modeling | Data PrivacySenior-level Full TimeUnited States; United States, Washington, Redmond; …19h ago
-
Data Scientist (Brooklyn Nets - Basketball Operations) USD 150K-170KAWS | Azure | Cloud platform | Computer Vision | D3.jsSenior-level Full TimeBrooklyn, NY 1123220h ago
-
Young Investigator, Open Language Models for Biology USD 159K-159KAnndata | Code Development | Controlled experiments | Data Analysis | Deep learningDental insurance | Employee assistance program | Family leave | Healthy work-life balance | Medical insuranceNone Full TimeSeattle, WA20h ago
-
Principal Data Scientist, GTM Data Science USD 182K-242KAWS SageMaker | Apache Spark | Boosted Trees | Causal Inference | DatabricksEmployee resource groups | Free home security system | Professional monitoringSenior-level Full TimeBoston, MA21h ago
-
Atomistic Simulation | Density Functional Theory | Electronic structure | First principles | Machine LearningCommuter benefits | Disability insurance | Educational assistance | Flexible time off | Life insuranceSenior-level Full TimeCambridge, MA USA22h ago
-
AI Safety | AI research | Data Analysis | Data acquisition | Evaluation metrics401k matching | Dental insurance | Flexible working arrangements | Life insurance | Medical insuranceSenior-level Full TimeEvanston, United States23h ago