LLM Evaluation Engineer

Remote R

USD 160K-255K (estimate) Senior-level Full Time

@ T...

Apply Save

Found 2d ago

Tasks

Apply rule based systems for policy enforcement
Build tools to observe debug and improve evaluator performance
Define reusable evaluation components for scalable use cases
Design real time evaluation logic for LLM outputs
Implement semantic similarity based evaluation strategies
Integrate evaluation outputs with downstream enforcement actions
Prototype tune and productize small language models and prompt templates
Run statistical checks for evaluation
Score prompts responses using foundation models

Perks/Benefits

Skills/Tech-stack

Education

N/A

Roles

Engineer | Evaluation Engineer | LLM Evaluation Engineer | Learning Engineer | Machine Learning Engineer

Apply Save

Language: en Views: 0

Clicks: 0

Saves: 0

Related jobs

🚀 Senior AI / Fullstack Engineer A USD 129K-170K

AI Evaluation | CI/CD | Cloud | DevOps | LLM

Paid in USD | Remote work

Senior-level Full Time

Guatemala, Guatemala (Remote) R

17h ago
Senior Electrical Engineer (BESS/Storage) (onsite, remote option) A USD 120K-178K

AC DC Sizing | AC/DC | Arc Flash | Artificial Intelligence | Automation

Multiple project extensions | Paid on time | Remote work option

Senior-level Full Time

Juno Beach, FL R

18h ago
AI Software Engineer A USD 100K-170K

AI Agents | API Development | Azure | CI/CD | LLMOps

Collaborative cross-functional team | Fully remote

Mid-level Full Time

Canada, Ontario; Toronto, None; Toronto, Ontario; … R

18h ago
Senior Solutions Architect / Sales Engineer, HPC & Computational Engineering USD 175K-200K

Benchmarking | C plus plus | CUDA | Computational Engineering | Distributed Computing

401k match | Generous paid time off | Medical, dental & vision coverage | Stock options | WFH Hardware Reimbursement

Senior-level Full Time

Sunnyvale, CA R

1d ago
Staff ML Engineer, Foundation Models USD 251K-310K

Bayesian Inference | Foundation Models | Generative Modeling | Hierarchical learning | Language Models

Bonus | Equity incentive

Senior-level Full Time

Mountain View, CA, USA R

1d ago
Information Security Engineer (Generalist – AI & Automation Focus) INR 1860K-3000K

AI Agents | AI Governance | API Security | Auditability | Cloud Security

Hybrid work | Onsite support

Mid-level Full Time

Bengaluru, Karnataka, India R

1d ago
AVP Software Engineer - Data Platform Engineering (Python/AWS) USD 127K-211K

API Gateway | AWS Lambda | AWS Step Functions | Amazon SQS | Apache Airflow

401k matching | Employee stock options | Health benefits | Hybrid work onsite 2 to 3 days per week | Paid time off

Executive-level Full Time

New York, United States R

1d ago
Senior Kubernetes Platform Engineer - AI/ML Infrastructure USD 139K-203K

AIOps | Anomaly Detection | Anthos | CRD | Control Plane

401k plan | Basic life insurance | Life insurance | Long-term disability | Medical/Dental/Vision insurance

Senior-level Full Time

Research Triangle Park, North Carolina, United … R

1d ago
Senior Machine Learning Engineer USD 160K-200K

AWS | CI/CD | Computer Vision | Data Processing | Docker

Commuter benefits | Dental insurance | Employee assistance program | Equity | Health insurance

Senior-level Full Time

Arlington, VA R

1d ago
Senior Computer Vision & Machine Learning Engineer USD 140K-206K

Ablation Studies | CI/CD | Data Pipelines | Docker | Error Analysis

Autonomy | Client project impact | Support for professional growth

Senior-level Full Time

Remote, US R

1d ago
Computer Vision & Machine Learning Engineer USD 150K-200K

Ablation Studies | CI/CD | Computer Vision | Data Pipelines | Docker

Entry-level Full Time

Remote, US R

1d ago
Data Engineer | CDMX | Híbrido MXN 480K-480K

Agile | Amazon EMR | Amazon Redshift | Amazon Web Services | Apache Airflow

Growth plan | International exposure | Legal benefits | Training and certifications

Entry-level Full Time

Mexico City, Mexico City, Mexico R

1d ago
Sr Staff AI Engineer - Veza USD 190K-334K

Agent Orchestration | Algorithms | C++ | Data Structures | Distributed Systems

401k plan with match | ESPP | Family leave programs | Flexible time away | Health plans

Senior-level Full Time

Minneapolis, Minnesota, United States R

1d ago
AI/ML Engineer USD 152K-220K

3D Reconstruction | 3D Scene | 3D Scene Understanding | AI Agents | Cloud Computing

Agile team collaboration | Fully remote | Production monitoring ownership

Mid-level Full Time

San Francisco, CA (Hybrid) or United … R

1d ago
Senior GTM AI Engineer USD 154K-213K

Agent Orchestration | Agent Performance | Agent Performance Monitoring | Anthropic Claude | App Development

401k match | Catered lunch | Company events | Desk setup stipend | Flexible spending accounts

Senior-level Full Time

Oakland, California, United States R

1d ago
Ingeniero Big Data MXN 480K-480K

Artifactory | CI/CD | Cloudera | ETL | Git

Hybrid work | Technical documentation

Senior-level Full Time

Ciudad de México, CDMX, Mexico R

1d ago
#Hckt 2026 - Dados (IA) BRL 23K-23K

AWS | Amazon Aurora | Databricks | LLMs | Language Processing

Entry-level Full Time

Brasil R

1d ago
Machine Learning Engineer CAD 110K-130K

Amazon Web Services | C++ | Cloud platform | Code Quality | Computer Vision

Employee wellness programs | Free on-site gym | Hybrid work option | Modern office | On-site daycare

Mid-level Temporary

Toronto, Ontario, CA R

1d ago
Data Science Engineer USD 121K-154K

Data Visualization | Documentation | Entity Resolution | Fine Tuning | Git

401k | Education reimbursement program | Flexible schedules | Hybrid schedule | Relocation assistance

Entry-level Full Time

Livermore, CA, United States R

1d ago
Senior Data Engineer CZK 1092K-1500K

AWS | Airflow | Amazon Redshift | Apache Spark | CI/CD

Coursera access | Employee referral rewards | Flexible time off | Paid public holidays | Paid sick leave

Senior-level Full Time

Remote - Czech Republic R

1d ago
Software Engineer, Applied AI Research USD 180K-400K

Backend architecture | Generative AI | Image Generation | Language Models | Large Language Models

Career growth potential | Remote-first work environment

Senior-level Full Time

Remote (North America) R

1d ago
AI Architect USD 120K-197K

AI orchestration | Automation | Cloud Computing | Data Governance | Distributed Systems

Weekly earned wage access option

Senior-level Full Time

Durant, OK, United States R

1d ago
DevOps / DataOps Engineer – Databricks Platform | Senior (Remote) BRL 168K-180K

AWS | Asset bundles | Azure | Bitbucket Pipelines | CI/CD

Remote work

Senior-level Full Time

Brasil R

1d ago
Staff Diagnostics and Embedded Cyber Security Engineer USD 150K-283K

AUTOSAR | Automotive Ethernet | BLE | C# | C++

Employee resource groups | Flexible family care days | Paid parental leave | Paid time off | Subsidized back-up child care

Senior-level Full Time

Palo Alto, CA, United States R

1d ago
Senior Analytics Engineer USD 153K-206K

AWS | Data Quality | ETL | Healthcare Claims | PySpark

401k plan | Commuter benefits | Dental insurance | Disability insurance | Flexible spending account

Senior-level Full Time

Remote- US R

1d ago