AIML - Sr Applied AI Scientist - GenAI Model Autograding, Evaluation
Tasks
- Adapt foundation models for evaluation
- Build evaluation methodology for quality safety and user experience
- Collaborate to deploy autograders in production
- Design prompting techniques for automated grading
- Develop autograder systems for AI quality evaluation at scale
- Diagnose autograder limitations and biases
- Synthesize findings into actionable insights
Perks/Benefits
- N/A
Skills/Tech-stack
Bias detection | Data Quality | Data quality analysis | Foundation Model | Foundation model adaptation | Human-in-the-loop | Image quality | Image quality evaluation | LLMOps | Language Models | Large Language Models | Model Adaptation | Model Evaluation | Monitoring | Prompt engineering | Quality analysis | Quality evaluation | The Loop
Education
Regions
Countries
States
Cities
Related jobs
-
Senior Data Scientist, GeminiApp, DeepMind USD 232K-252KData Analysis | Data Modeling | Experiment design | Language Processing | Machine LearningSenior-level Full TimeSan Francisco, CA, USA; Mountain View, …4h ago
-
Causal Inference | Data Exploration | LLM | Language Models | Large Language ModelsSenior-level Full TimeMountain View, CA, USA; New York, …4h ago
-
Research Data Scientist, Magic Eye USD 147K-210KCausal Inference | Data Modeling | Data Quality | Experimentation | PythonMid-level Full TimeMountain View, CA, USA4h ago
-
Senior Data Scientist USD 169K-190KA/B | A/B Testing | B testing | Dashboards | Data AnalysisDisability benefits | Health insurance | Life insurance | Paid time off | Parental leaveSenior-level Full TimeSanta Monica, California R11h ago
-
AI Research Scientist - Optexity USD 150K-250KBenchmarking | Computer use | Data pipeline | Deep learning | Language ModelsVisa sponsorshipMid-level Full TimeSan Francisco Bay Area12h ago
-
Staff Data Scientist USD 172K-209KAmazon Bedrock | Amazon SageMaker | Claude | Cloud Data | Cloud Data Science PlatformsSenior-level Full TimeUS - Remote ; Canada - … R18h ago
-
Mid-level Full TimeUnited States18h ago
-
Principal Data Scientist - FDE USD 142K-274KData Ethics | Data Privacy | Data Science | Debugging | ExperimentationSenior-level Full TimeUnited States; United States, Washington, Redmond; …18h ago
-
Senior Data Scientist - FDE USD 119K-234KData Ethics | Data Modeling | Data Privacy | Data acquisition | DebuggingSenior-level Full TimeUnited States; United States, Washington, Redmond; …18h ago
-
Staff Applied Scientist USD 300K-390KAWS | Experimentation | Heuristics | Huggingface | Language Models401k match | Flexible PTO | Flexible spending accounts | Medical/Dental/Vision | Teladoc HealthSenior-level Full TimeNew York City, New York19h ago
-
Principal Applied Scientist USD 165K-296KArtificial Intelligence | Calibration | Data strategy | Deep learning | Distributed TrainingSenior-level Full TimeUnited States, Washington, Redmond; United States, …19h ago
-
APIs | Anomaly Detection | Audit Analytics | Automated testing | Continuous AuditingBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site wellness centersEntry-level Full TimePlano, TX, United States19h ago
-
Data Scientist I USD 119K-156KAnomaly Detection | Data Modeling | Data Pipelines | Data Quality | Data Validation401k plan participation | Annual performance bonuses | Company equity grants | Life and disability insurance | Medical, dental, and vision benefitsEntry-level Full TimeNew York, New York, United States R21h ago
-
Lead Data Scientist, AI Labs USD 164K-201KAWS | Algorithmic Bias Mitigation | Algorithmic bias | Automatic Speech Recognition | AzureHybrid work | Paid time offSenior-level Full TimeWashington, District of Columbia, United States R21h ago
-
Principal Data Scientist USD 188K-304KCloud Computing | Data Analysis | Data Ethics | Data Management | Data PrivacySenior-level Full TimeUnited States1d ago
-
Staff Machine Learning Scientist USD 182K-273KA/B | A/B Testing | Agentic AI | B testing | Cloud Computing401 k retirement plan | Employee stock purchase plan | Generous time off | Healthcare benefits | Mental health or EAP programsSenior-level Full TimeUnited States R1d ago
-
Principal ML Scientist USD 215K-323KAI-assisted software development | Agent systems | Agentic AI | Cloud Platforms | ComplianceEmployee stock purchase plan | Generous time off | Healthcare benefits | Life insurance | Long-term disabilitySenior-level Full TimeUnited States R1d ago
-
Data Scientist Associate USD 77K-126KClinical Practice | Clinical Research | Clinical data | Clinical data management | Clinical research dataHybrid eligible | Training providedMid-level Full TimeUniversity R1d ago
-
Senior Fellow - Statistics and Data Science USD 200K-230KArea Estimation | Bayesian analysis | Causal Inference | Data Quality | Data VisualizationDental insurance | Employee Assistance Program (EAP) | Health insurance | Life insurance | Long-term disability insuranceSenior-level Full Time Part TimeIllinois, Chicago - 300 E Randolph … R1d ago
-
Principal Data Scientist, Google Search USD 307K-427KAI | Data Science | Information Retrieval | Machine Learning | Measurement frameworksSenior-level Full TimeMountain View, CA, USA; Kirkland, WA, …1d ago
-
Data Scientist USD 180K-200KData Manipulation | Data Science | Healthcare Data | Hugging Face | Language ModelsSenior-level Full TimeBoston or NYC1d ago
-
Computer/Data Scientist USD 120K-145KAsynchronous programming | Clustering | Containerization | Deep learning | EmbeddingsTelework optionMid-level Full TimeWest Mifflin, PA, United States R1d ago
-
Associate Director, Applied AI USD 135K-176KAgentic Systems | Artificial Intelligence | Data Science | Language Models | Large Language ModelsRegular travelMid-level Full TimeUnited States1d ago
-
Senior Data Scientist USD 115K-189KABI Tradecraft | Activity Based Intelligence | Artificial Intelligence | Classification | Data Preprocessing401k matching | Medical, dental & vision coverage | Paid Holidays | Paid time off | Pet insuranceSenior-level Full TimeArlington, VA1d ago
-
Senior Data Scientist / ML Engineer (Gen AI)_Boston USD 60K-210KA/B | A/B Testing | AI Foundry | AI Governance | AWS Bedrock401k retirement plan | Medical, dental, and vision benefits | Paid Holidays | Paid time off | Variable pay and incentivesSenior-level Full TimeUnited States1d ago