AIML - Sr Applied AI Scientist - GenAI Model Autograding, Evaluation
Tasks
- Adapt foundation models for evaluation
- Build evaluation methodology for quality safety and user experience
- Collaborate to deploy autograders in production
- Design prompting techniques for automated grading
- Develop autograder systems for AI quality evaluation at scale
- Diagnose autograder limitations and biases
- Synthesize findings into actionable insights
Perks/Benefits
- N/A
Skills/Tech-stack
Bias detection | Data Quality | Data quality analysis | Foundation Model | Foundation model adaptation | Human-in-the-loop | Image quality | Image quality evaluation | LLMOps | Language Models | Large Language Models | Model Adaptation | Model Evaluation | Monitoring | Prompt engineering | Quality analysis | Quality evaluation | The Loop
Education
Regions
Countries
States
Cities
Related jobs
-
Data Scientist USD 75K-127KAI Search | Agile | Azure | Azure AI | Azure AI SearchDisability insurance | Health insurance | Holiday pay | Learning and development | Life insuranceMid-level Full TimeUSA-VA-Arlington, USA-Remote Work6h ago
-
Data Scientist USD 174K-211KData Analysis | Data Quality | Data Visualization | Dataset Preparation | ExperimentationBenefits determined by role level and location | Bonus | Hybrid scheduleMid-level Full TimeSan Bruno, CA, USA R9h ago
-
Data Scientist USD 177K-198KData Manipulation | Data Quality | Data Security | Data extraction | Descriptive StatisticsBenefits determined by role level and location | Hybrid work scheduleMid-level Full TimeSunnyvale, CA, USA R9h ago
-
Research Scientist, Cloud AI Research USD 147K-210KAdversarial Training | C++ | DPO | Deep learning | Foundation ModelsSenior-level Full TimeSunnyvale, CA, USA9h ago
-
APIs | Agentic AI | Cloud Computing | Data Modeling | Data Pipelines401k matching | Company holidays | Dental insurance | Elder and Childcare Days | Employee stock purchase programMid-level Full TimeNeedham, MA, United States19h ago
-
Data Scientist - Expert - #914 USD 130K-193KAdvanced Analytics | Artificial Intelligence | Cloud Computing | Containerization | Data Engineering100 percent onsite | Security clearance TS SCISenior-level Full TimeDayton, Ohio, United States20h ago
-
Senior Data Scientist, Biologics Discovery USD 109K-174KCloud Computing | Data Leakage Risk | Data leakage | Dataset curation | Distribution ShiftSenior-level Full TimeUS026 PA Spring House - 1400 …20h ago
-
Activation Patching | Bayesian statistics | Causal tracing | JAX | Language ModelsSenior-level Full TimeSeattle22h ago
-
Applied AI Scientist - Commercial USD 190K-225KAmazon Bedrock | Amazon SageMaker | Causal Inference | Data Quality | DatabricksBonus or performance based award within 12 months | Model monitoring and remediation support | Short-term incentive planMid-level Full TimeBridgewater, NJ, United States23h ago
-
Advanced Data Scientist USD 87K-99KAgile Development | Amazon Web Services | Artificial Intelligence | Azure | CRM Analytics12 paid holidays | 401k match | Dental insurance | Educational assistance | Employee assistance programMid-level Full TimeHamilton, NJ, United States R1d ago
-
Associate Data Scientist/ Data Scientist USD 70K-118KAI Services | ARIMA | AWS | Azure AI | Azure AI ServicesMid-level Full TimeVienna, VA, United States1d ago
-
Applied Scientist USD 119K-234KAI Evaluation | Dashboards | Data Mining | Data Pipelines | Experiment designSenior-level Full TimeUnited States, Washington, Redmond; United States, …1d ago
-
Senior Data Scientist - Product USD 169K-228KA/B | A/B Testing | B testing | Causal Inference | Cohort AnalysisSenior-level Full TimeNew York City1d ago
-
Lead AI Engineer USD 110K-155KAIOps | Azure DevOps | Azure OpenAI | CI/CD | Databricks401k company match | Dental insurance | Employee assistance program | Employee pricing program | Health insuranceSenior-level Full TimeOrlando, FL, United States1d ago
-
Lead AI Engineer USD 125K-180KAPIs | AWS | AWS Bedrock | Agentic Workflows | Azure401k | Commuter benefits | Dental insurance | Disability coverage | Employee assistance programSenior-level Full TimeColumbia, MD, United States1d ago
-
Data Scientist, Machine Learning USD 180K-200KAPI Development | CI/CD | Data Monitoring | Data Quality | Data Validation401k | Medical, dental, and vision insurance | Paid time offMid-level Full TimeGreenwich, CT1d ago
-
AI Architect & RAG Data Science Engineer USD 147K-206KAccess Management | Azure | Azure DevOps | BM25 | C#Onsite collaboration | Security clearance support | Telework optionsSenior-level Full TimeHuntsville, AL, USA1d ago
-
Statistician I - Biostatistics & Data Science USD 80K-120KBioinformatics | Clinical Trial Design | Clinical trial | Computational Genomics | Data QualityDental insurance | Health insurance | Paid time off | Retirement plan | Vision insuranceMid-level Full TimeRobinson L, United States1d ago
-
Data Scientist / Principal Investigator USD 108K-227KAPIs | Artifact management | CI/CD | Computer Vision | Data PipelinesFlexible time off | Healthcare benefits | Learning and development | Relocation assistance | Retirement benefitsSenior-level Full Time437 DENVER CO, United States1d ago
-
Data Scientist USD 123K-191KAI Agents | Algorithms | Classification | Clustering | Data Structures401k | Dental insurance | Life insurance | Medical insurance | Paid time offMid-level Full TimeAustin, TX, United States R1d ago
-
Data Analytics & Management, VP USD 120K-202KAgentic AI | Anomaly Detection | Cause analysis | Classification | Data GovernanceEducational support | Employee networks | Flexible work programs | Insurance | Medical careExecutive-level Full TimeBOSTON, United States1d ago
-
Data Analytics & Management, AVP USD 90K-157KAlteryx | Anomaly Detection | Classification | Data Modeling | Data QualityExecutive-level Full TimeBOSTON, United States1d ago
-
Sr Data Scientist - Advanced AI / AI Factory USD 98K-176KAPI Integration | Deep learning | Embeddings | Experiment design | Feature Engineering401k | Employee discount | Health benefits | Long-term disability | Paid national holidaysSenior-level Full Time7000 Target Pkwy N,NCD-0375 Brooklyn Park,MN … R1d ago
-
Mid-level Full TimeUSA - New York - 1633 …1d ago
-
Technical Intern (Masters or PhD) USD 120K-120KDeep learning | Language Models | Language Processing | Large Language Models | Natural LanguageEntry-level Internship Part TimeRemote Work( USA), United States R1d ago