AI Evaluation Scientist
Tasks
- Analyze model behavior
- Assess AI model outputs
- Build evaluation scripts
- Collaborate with data scientists
- Contribute to evaluation framework development
- Design evaluation processes
- Develop benchmark datasets
- Develop test harnesses
- Document evaluation results
- Perform error analysis
- Support responsible deployment
Perks/Benefits
Skills/Tech-stack
AI Evaluation | AI evaluation frameworks | Behavior Analysis | Data Analysis | Evaluation Frameworks | Evaluation metrics | Hugging Face | Langchain | Language Processing | Model behavior | Model behavior analysis | Natural Language | Natural Language Processing | PyTorch | Python | Scikit-learn | Statistical Testing | Test Design
Education
Roles
Related jobs
-
Junior Data Scientist - Government & Public Services USD 82K-110KCloud Computing | Data Mining | Data Visualization | Language Models | Large Language ModelsEntry-level Full TimeArlington/Rosslyn, Virginia, United States1h ago
-
Entry-level Full TimeMenlo Park, CA3h ago
-
AI Research Scientist - FAIR Social Intelligence USD 177K-251KArtificial Intelligence | Behavioral Science | Game theory | Machine Learning | PythonMid-level Full TimeBellevue, WA3h ago
-
AI Research Scientist - FAIR Social Intelligence USD 177K-251KArtificial Intelligence | Game theory | Machine Learning | Python | Reinforcement LearningEntry-level Full TimeBellevue, WA | Seattle, WA3h ago
-
AI Research Scientist, FAIR Chemistry USD 117K-173KApplied Mathematics | Artificial Intelligence | Computational statistics | Generative Models | Machine LearningEntry-level Full TimeSan Francisco, CA3h ago
-
AI Research Scientist, FAIR Chemistry USD 147K-208KApplied Mathematics | Artificial Intelligence | Computational statistics | Generative Models | Machine LearningSenior-level Full TimeMenlo Park, CA | San Francisco, …3h ago
-
Research Scientist Intern, PyTorch Compiler (PhD) USD 93K-142KCUDA | Distributed Training | Learning systems | ML Compiler | Machine LearningEntry-level InternshipMenlo Park, CA | Seattle, WA …3h ago
-
Research Scientist, AI & Systems Co-design (PhD) USD 117K-173KAI hardware | C++ | Distributed Systems | Hardware Architecture | ML frameworksMid-level Full TimeMenlo Park, CA3h ago
-
Research Scientist Intern, Robot Foundation Models (PhD) USD 108K-200KBehavioral models | Control Theory | Dynamics | Imitation Learning | JAXEntry-level InternshipRedmond, WA | Burlingame, CA3h ago
-
Research Scientist, AI Networking (PhD) USD 120K-230KCUDA | Distributed ML | Distributed ML training | GPU Architecture | High PerformanceEntry-level Full TimeMenlo Park, CA3h ago
-
AI research | Computer Vision | Data Curation | Language Processing | Model TrainingSenior-level Full TimeMenlo Park, CA3h ago
-
Product Data Scientist, G1 and Photos USD 138K-198KData Analysis | Machine Learning | Python | R | SQLFlexible work hours | Health insurance | Professional developmentMid-level Full TimeSan Francisco, CA, USA; Mountain View, …3h ago
-
Product Data Scientist, Search Verticals and Translate USD 138K-198KCausal Inference | Data Science | Experimentation | Forecasting | Machine LearningBenefits | Bonus | EquityMid-level Full TimeMountain View, CA, USA3h ago
-
Data Analysis | Information science | Machine Learning | Python | Quantum InformationBenefits | Health insurance | Professional developmentMid-level Full TimeGoleta, CA, USA3h ago
-
AI Engineer USD 118K-177KData Analysis | Distributed Computing | Machine Learning | Optimization | PythonBenefits package | Flexible work arrangementsEntry-level Full TimeJuno Beach, FL, US, 334087h ago
-
Senior AI/ML Engineer USD 125K-157KAI platforms | AWS | Agentic AI | Agentic AI Platforms | Data GovernanceDental insurance | Disability plans | Employee referral program | Fertility benefits | Life insuranceSenior-level Full TimeUS Remote R8h ago
-
AI Program Manager USD 125K-165KAI Technologies | Agile methodology | Automation | Business Analysis | Change Management401k benefits | Career development opportunities | Flexible work | Performance incentives | Spot AwardsMid-level Full TimeBoston, MA, United States10h ago
-
AI Engineer USD 140K-180KAI Automation | APIs | CI/CD | Data integration | JavaScriptCollaborative environment | Flexible working hours | Growth opportunities | Innovative projects | Remote work optionsMid-level Full TimeNew York, New York10h ago
-
AI Application Engineer USD 80K-176KAI APIs | Agile methodologies | Application development | CI/CD | Cloud Platforms401k match | Company events | Holidays | Life insurance | Medical/Dental/Vision insuranceMid-level Full TimeSt. Louis, MO, US11h ago
-
AI SME / Developer USD 104K-155KAI ethics | AWS SageMaker | Azure ML | Bias Mitigation | Data EngineeringHealthcare benefits | Hybrid work environment | Paid training | Travel opportunitiesMid-level Full TimeColumbia, MD, US11h ago
-
Bioinformatics Scientist/Senior Bioinformatics Scientist USD 124K-193KAWS | Bioinformatics | Docker | Genomics | GitDental insurance | Disability insurance | Health insurance | Life insurance | Time offSenior-level Full TimeWaltham, MA, US13h ago
-
Applied Scientist, Demand Forecasting USD 142K-193KDeep learning | Distributed Systems | Generative AI | Graph Neural Networks | Neural NetworksMid-level Full TimeBellevue, Washington, USA14h ago
-
ML Architect USD 152K-202KAI Governance | APIs | Cloud Native | Distributed Systems | EmbeddingsCollaborate with Fortune 500 | Impact at AI-first company | Upskill opportunities | Work with innovative technologySenior-level Full TimeUSA - Remote, United States R14h ago
-
AI Safety | Cloud Platforms | Containerization | Deep learning | Distributed Systems401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States14h ago
-
Principal, Data Science & Analytics USD 139K-304KA/B | A/B Testing | B testing | Big Data | Data AnalysisFlexible work hours | Health insurance | Professional development | Remote work optionsSenior-level Full TimeRedmond, WA, US; Mountain View, CA, …14h ago