Applied AI Researcher, Agent Systems & Evaluation
Tasks
- Assess model trustworthiness of evaluators
- Build evaluation pipeline for AI agents
- Collect eval data from production traces
- Construct automated eval loop for every change
- Design task suites that reflect real distribution
- Design verifier guided selection and escalation strategies
- Prevent task suite gaming
- Read frontier research and convert into live experiments
- Run automated hill climbing for prompt context tools and routing
- Run test time scaling experiments
- Set noise floors and acceptance statistical standards
- Supervise fine tune vision language models
- Train with reinforcement learning
Perks/Benefits
- Direct access to compute
- Hands on research with production data
- Work with CEO
- Work with engineering leadership
Skills/Tech-stack
Artificial Intelligence | Evaluation Data | Experiment design | Fine Tuning | Human Feedback | Inference Optimization | Language Models | Large Language Models | Machine Learning | Machine Learning Ops | Model Evaluation | Prompt engineering | Reinforcement Learning | Sampling strategies | Search Algorithms | Statistical Analysis | Supervised Fine Tuning | Task automation | Verifier Systems | Vision Language Models | Vision-language
Education
Bachelor of Science | Doctor of Philosophy | Master of Science
Regions
Countries
States
Related jobs
-
AI Native Engineer USD 130K-240KAI Agents | API Integration | Anthropic | Backend Development | Cloud infrastructureFlexible independent work | In person working sessionsMid-level Full TimeNew York, NY10h ago
-
Senior AI Engineer USD 144K-236KA/B | A/B Testing | B testing | Classification | Data MiningFlexible work arrangement | Health and wellness programs | Time off for employeesSenior-level Full TimeMountain View, CA, United States R12h ago
-
A/B | A/B Testing | B testing | CSAT | ChatbotRemote workMid-level Full TimeOak Brook, IL, United States15h ago
-
AI Solutions & Innovation Manager USD 78K-90KAPI Integration | Accessibility | Agent Development | Artificial Intelligence | Change ManagementAnnual leave | Paid Holidays | Paid parental leave | Professional development | Public service loan forgiveness programMid-level Full TimeRichland County15h ago
-
Sr. Dir., AI Engineering USD 251K-385KAI Platform | AI Security | API first | API-first architecture | AWS401k match | ESPP | Family leave programs | Flexible spending accounts | Flexible time awaySenior-level Full TimeSanta Clara, CALIFORNIA, United States R16h ago
-
Senior-level Full TimeAnnapolis Junction, United States R18h ago
-
Entry-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL18h ago
-
Palantir Engineer - AI and Data Science Engineer III USD 131K-218KArtificial Intelligence | Data Engineering | Data Modeling | Data Pipelines | Data TransformationActive secret clearanceSenior-level Full TimeHuntsville, Alabama, United States19h ago
-
Visual Generation & Multimodal Evaluation Machine Learning Engineer Graduate (AML-Ark-US) - 2027 Start USD 128K-256KAutomated metrics | Computer Vision | Data Engineering | Data Pipelines | Deep learningEntry-level Full TimeSan Jose, California, United States20h ago
-
Research Intern (AI/LLM Network) - 2026 Start (PhD) USD 136K-300KDistributed Systems | Linux | Machine Learning | Network Protocols | NetworkingEntry-level InternshipSan Jose, California, United States20h ago
-
Postdoctoral Researcher, AI and Systems Co-design Team USD 112K-145KAI accelerator | AI infrastructure | Compilers | Computer Architecture | Distributed SystemsEntry-level Full TimeMenlo Park, CA | Remote, US R21h ago
-
Quantitative UX Researcher, Search Ads USD 132K-189KC++ | Experimental Design | Go | Hypothesis Testing | JavaInternal tools | Mentorship | Regular meetupsMid-level Full TimeNew York, NY, USA; Irvine, CA, …21h ago
-
Senior Product Quantitative Researcher, Cloud Security USD 159K-230KBigQuery | Cloud platform | Data Analysis | Data Pipelines | Distributed ComputingInternal tools | Meetups | MentorshipSenior-level Full TimeSeattle, WA, USA; Sunnyvale, CA, USA21h ago
-
Computer Vision | Data labeling | Error Analysis | Geometric transformations | HomographyRemote workMid-level FreelanceUnited States of America23h ago
-
Senior/Staff - Agent Platform (Cortex Code) USD 236K-309KApache Airflow | DBT | Data Analysis | Data Engineering | Data ModelingSenior-level Full TimeUS-CA-Menlo Park1d ago
-
AI & Data Intern USD 20K-24KClassification | Clustering | Data Analysis | Data Pipelines | Data VisualizationMentorshipEntry-level InternshipPleasant Hill, IA, US1d ago
-
Tech Lead Manager, Cortex AI - Snowflake Intelligence USD 200K-287KData Modeling | Experimentation | Function Calling | Go | Information RetrievalSenior-level Full TimeUS-CA-Menlo Park1d ago
-
Lead AI/ML Engineer - Remote USD 145K-249KAWS | Agentic AI | Artifact management | Azure | CI/CD401k contribution | Comprehensive benefits package | Equity stock purchase | Incentive and recognition programs | Remote work flexibilitySenior-level Full TimeSan Francisco, California, UNITED STATES; REMOTE R1d ago
-
Associate AI Engineer USD 80K-146KAI orchestration | API Integration | Agent SDK | Agile | Fine Tuning401k | Dental insurance | Employee stock purchase plan | Flexible spending account | Medical insuranceMid-level Full TimeWA-Headquarters, Building 5, United States1d ago
-
AI Engineer II USD 60K-90KAPI Development | Backend Development | Cloud Computing | Databricks | DockerNo sponsorship | Onsite workMid-level Full TimeSSC Irving TX, United States1d ago
-
AI Engineer USD 50K-112KAWS | Adversarial Benchmarking | Cloud Computing | Databricks | Deep learningEntry-level Full TimeTPA ESC-4040 W Boy Scout Blvd, …1d ago
-
AI Engineer Internship USD 50K-50KData Pipelines | Deep learning | Fine Tuning | Generative AI | GitPaid internshipEntry-level InternshipDallas, TX R1d ago
-
AI Software Developer, Junior USD 55K-126KAI | APIs | Automation | Debugging | EmbeddingsDependent care | Mentorship | Paid leave | Professional development | Tuition assistanceEntry-level Full TimeUSA, CA, San Diego (49275 Electron … R1d ago
-
AI Software Developer, Senior USD 86K-198KAI | API Integration | Agile | Automation | Cloud deploymentDependent care | Disability insurance | Health benefits | Life insurance | Paid leaveSenior-level Full TimeUSA, CA, San Diego (49275 Electron … R1d ago
-
Managed Services - Data, Analytics & AI - Manager USD 99K-232KAI Model Monitoring | AI model | AI pipeline | Artificial Intelligence | BudgetingSenior-level Full TimeDallas - 2121 North Pearl Street, …1d ago