Machine Learning Eval Engineer

San Francisco, CA; Onsite R

A USD 150K-300K Senior-level Full Time

@ R...

Apply Save

Found 20h ago

Tasks

Build customer specific benchmarks for real world workflows
Build internal tooling for model visualization and evaluation analysis
Build workflows to identify failure modes across datasets
Design and build scalable evaluation systems for production LLM applications
Design evaluation infrastructure for production AI systems
Design statistical evaluation methodologies
Develop benchmarks metrics and automated evaluation pipelines
Own evaluation systems from design through production deployment
Partner with ML engineers to prioritize model improvements using evaluation insights
Prototype evaluation techniques using LLM as a judge
Work with unstructured documents including PDFs spreadsheets and OCR outputs

Perks/Benefits

Skills/Tech-stack

Education

Bachelor of Engineering | Bachelor of Science | Master of Science | PhD

Roles

Apply Save

Language: en Views: 0

Clicks: 0

Saves: 0

Related jobs

Featured Feat. Associate Director, Data Labs USD 167K-167K

AWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM Governance

Conference speaking opportunities | Hybrid work schedule | Media appearances

Senior-level Full Time

Washington, District of Columbia, 20004, United … R

5d ago
Computational Designer USD 95K-118K

C# | C++ | Computational Geometry | Computer Graphics | Data Pipelines

401k plan | Dental insurance | Education assistance | Fertility support | Flexible time off

Mid-level Full Time

Portland, OR, US R

14h ago
Senior Software Engineer, Data Foundation USD 189K-256K

Django | Go | High Volume | High-volume APIs | Python

401k | Enhanced parental leave | Generous vacation | Holiday Breaks | Medical, dental & vision coverage

Senior-level Full Time

San Francisco, US (Hybrid) R

20h ago
Principal Machine Learning Engineer USD 190K-305K

Deep learning | Experimentation | Feature Engineering | Information Retrieval | LLM

Health and wellbeing resources | Paid volunteer days

Senior-level Full Time

Singapore - Singapore - Central Singapore … R

20h ago
Software Engineer II - Data Platform USD 116K-162K

Airflow | BigQuery | DBT | Docker | Go

Flexible time off | In-office workspace | LinkedIn Learning access | Medical insurance | Paid Holidays

Mid-level Full Time

United States (Remote) R

21h ago
Senior Data Engineer - AWS Data Platform (Redshift, Glue, and Airflow) USD 100K-115K

APIs | AWS Glue | AWS Redshift | Amazon Web Services | Apache Airflow

Fully remote | Health insurance | On call production support | Paid time off | Retirement plans

Senior-level Full Time

Orlando, FL, United States R

23h ago
Analytics Lead USD 160K-240K

A/B | A/B Testing | Agent architecture | Airtable | B testing

401k | Dental insurance | Medical insurance | Vision insurance

Senior-level Full Time

Redwood City, CA (Hybrid) R

1d ago
Machine Learning / Data Science Engineer USD 90K-200K

Agent Frameworks | Agent Orchestration | Agentic AI | Amazon RDS | Automation

1 1 care | 401k matching | Carrot Fertility | Company paid stipend program | Disability insurance

Senior-level Full Time

Reston, VA, United States R

1d ago
Machine Learning / Data Science Engineer USD 90K-200K

Agent Orchestration | Agentic AI | Amazon RDS | Apache Spark | Automation

Disability insurance | Employer 401K matching | Fertility coverage | Flexible work environment | Generous PTO

Senior-level Full Time

Richmond, VA, United States R

1d ago
Lead Machine Learning / Data Science Engineer USD 90K-200K

Agent Orchestration | Agentic AI | Amazon RDS | Apache Spark | Azure SQL

401k matching | Carrot fertility benefits | Disability insurance | Employee resource groups | Flexible work environment

Senior-level Full Time

Reston, VA, United States R

1d ago
Lead Machine Learning / Data Science Engineer USD 90K-200K

Agent Orchestration | Agentic AI | Amazon RDS | Automation | Azure SQL

401k matching | Disability insurance | Employee resource groups | Fertility coverage | Generous paid time off

Senior-level Full Time

Richmond, VA, United States R

1d ago
Applied AI Engineer USD 160K-160K

Asynchronous task management | CI/CD | Context Management | Django | FastAPI

401k match | Medical insurance | Paid parental leave | Paid time off | Remote-first work environment

Senior-level Full Time

Remote (anywhere in the U.S.) R

1d ago
[2026] Senior Machine Learning Engineer (Systems), Embodied AI/NPCs, ML Platform - PhD Early Career USD 196K-243K

AWS | Azure | Cloud platform | Continuous batching | Data Pipelines

Equity compensation | Health benefits | Paid time off

Senior-level Full Time

San Mateo, CA, United States R

1d ago
[2026] Senior Machine Learning Engineer (Systems), Embodied AI/NPCs, ML Platform - PhD Early Career USD 196K-243K

AWS | Azure | Cloud platform | Continuous batching | Deep learning

Senior-level Full Time

San Mateo, CA, United States R

1d ago
Lead Embedded Linux Engineer - Intercom USD 210K-300K

Bash | C# | C++ | Device Drivers | Distributed Systems

Commuter benefits | Dental insurance | FSA | Flexible PTO | HSA

Senior-level Full Time

San Mateo, CA United States R

1d ago
Data Engineer, Engineering & Operations USD 115K-145K

Access Control | Aggregation Thresholds | Airflow | Alerting | Anonymization

401k | Dental insurance | Discounts | Fully remote | Medical insurance

Mid-level Full Time

New York, NEW YORK, United States R

1d ago
Sr. Databricks Consultant USD 175K-250K

AWS | Amazon S3 | Apache Spark | Azure | Azure Data

Remote work | Travel as needed

Senior-level Full Time

Work from home, VA, United States R

1d ago
Senior Data Engineer (AWS, Azure, GCP) USD 90K-200K

AWS | Amazon Aurora | Amazon Redshift | Azure | Azure Data

401k matching | Disability insurance | Fitness and travel stipends | Flexible work environment | Generous PTO

Senior-level Full Time

Reston, VA, United States R

1d ago
Senior Data Engineer (AWS, Azure, GCP) USD 90K-200K

AWS | Aurora | Azure | Azure Data | Azure Data Factory

401k matching | Disability insurance | Employee resource groups | Fertility coverage | Generous PTO

Senior-level Full Time

Richmond, VA, United States R

1d ago
Data Engineer (AWS, Azure, GCP) USD 90K-200K

AWS | Azure | BigQuery | Cloud platform | Databricks

1 to 1 mental health care | 401k matching | Certification support | Digital On Demand Learning | Disability insurance

Senior-level Full Time

Reston, VA, United States R

1d ago
Data Engineer (AWS, Azure, GCP) USD 90K-200K

AWS | AWS Lambda | Amazon EMR | Amazon Redshift | Aurora

1 on 1 mental health support | 401k matching | Disability insurance | Employee resource groups | Employer paid stipend program

Senior-level Full Time

Richmond, VA, United States R

1d ago
Senior Site Reliability Engineer, Robotics & Cloud Infrastructure USD 164K-220K

AWS | Alerting | Bash | Docker | Go

Employee benefits | Equity

Senior-level Full Time

Remote R

1d ago
ML Engineer, Manipulation USD 142K-193K

Behavior Cloning | Behavior Modeling | Computer Vision | Critical Systems | Data Augmentation

Mid-level Full Time

Anywhere in the US R

1d ago
HPC Scientific Software Engineer (IT@JH Research Computing) USD 85K-149K

Ansible | C plus plus | C# | CMake | CUDA

Mid-level Full Time

Baltimore, MD, United States R

1d ago
Senior AI Engineer | Sage Home Loans USD 150K-220K

Agent Orchestration | DPO | Databricks | Fine Tuning | Golden datasets

401k match | Disability insurance | Employee assistance program | Flexible PTO | Flexible spending accounts

Senior-level Full Time

Charlotte, NC R

1d ago

Machine Learning Eval Engineer

Tasks

Perks/Benefits

Skills/Tech-stack

Education

Roles

Regions

Countries

States

Cities

Related jobs