43 jobs for LLM-as-judge

Senior AI Engineer A BRL 240K-300K

Anthropic | Benchmarking | CI/CD | Docker | FastAPI

Autonomy | Fully remote | PJ contract structure

Senior-level Full Time

Brazil R

2d ago

Senior WEM AQM Prompt Engineer USD 88K-247K

A/B | A/B Testing | API Integration | B testing | Calibration

Inclusive environment | Remote work | Team-first culture

Senior-level Full Time

United States (Remote) R

2d ago

Senior AI Backend Engineer - Agent Evaluation & Quality USD 175K-250K

API Design | CI/CD | Cost Optimization | Experimentation | Function Calling

Senior-level Full Time

Makkah, Makkah Province, Saudi Arabia - … R

5d ago

DS / ML Engineer INR 1800K-3000K

Data Pipelines | Drift Detection | Inference Optimization | Information Retrieval | LLM

Mid-level Full Time

Bangalore

8d ago

Senior Machine Learning Engineer, Ads Content Understanding USD 216K-303K

Batch Processing | Bias Mitigation | C++ | Data Processing | Experimentation

401k employer match | Caregiving support | Family planning support | Flexible vacation | Gender-affirming care

Senior-level Full Time

Remote - United States R

10d ago

Agentic AI Architect A USD 125K-200K

.NET | AI Foundry | AI Search | Application Insights | Azure AI

Senior-level Full Time

New York, New York, United States

10d ago

Agentic AI Scientist - Office of Data Science USD 86K-154K

A2A | Agent Orchestration | Agents SDK | Auth0 | Automated testing

Senior-level Full Time

Memphis, TN, United States

11d ago

Senior Data Scientist, Applied AI USD 138K-230K

Agent evaluation | Data Pipelines | Deep learning | Experimentation | FastAPI

Senior-level Full Time

San Francisco, CA

13d ago

Machine Learning Lead Engineer USD 134K-224K

AI ethics | AWS AgentSquad | AWS Strands | Agent Frameworks | Audit trails

Additional paid time off options | Hybrid remote work | Paid Holidays | Paid vacation | Paid wellness time

Senior-level Full Time

Atlanta, GA - 6305 Peachtree Dunwoody … R

15d ago

AI Engineer - Automotive AI Systems USD 180K-269K

Adversarial Robustness | BLEU | Bias analysis | CI/CD | Dataset curation

Senior-level Full Time

Auburn Hills, MI, United States

15d ago

Sr. Software Engineer, GenAI USD 175K-220K

AWS | CI/CD | Docker | Drift monitoring | Embeddings

401k | Company laptop | Company retreats | Generous PTO | Health, dental, and vision insurance

Senior-level Full Time

Remote - USA R

15d ago

Staff Security Software Engineer, AI Security USD 190K-267K

AI Security | AI security tooling | AWS | Adversarial ML | Audit Logging

Senior-level Full Time

Remote - California R

16d ago

Product Data Scientist — AI Evaluation & Quality (Remote) EUR 56K-79K

Claude Code | Data Visualization | Databricks | DeepEval | Hypothesis Testing

Continuous professional development | Hybrid work | Relocation support | Remote work | Stock options

Senior-level Full Time

Athens R

17d ago

Product Data Scientist — AI Evaluation & Quality EUR 56K-79K

Databricks | Deep learning | Hypothesis Testing | LLM-as-judge | Python

Corporate apartment stay | Hybrid work | Remote work option | Stock options | Wellbeing support

Senior-level Full Time

Berlin R

20d ago

Product Data Scientist — AI Evaluation & Quality PLN 174K-321K

Agent evaluation | Classification | Claude Code | Databricks | DeepEval

EU travel flexibility | Personal development opportunities | Remote/work hybrid options | Stock options program | Supportive company culture

Senior-level Full Time

Warsaw R

20d ago

Senior AI Engineer THB 4000K-5000K

AWS Bedrock | Anthropic | Authentication | Azure OpenAI | CI/CD

Senior-level Full Time

Nawamin Road, Bangkok, Thailand

22d ago

Principal Engineer - Architect (AI Agents) A USD 177K-220K

AI Agents | AWS | Agent memory | Agent workflows | Azure

High autonomy | Remote/hybrid work

Senior-level Full Time

LATAM

22d ago

Senior Applied AI Engineer AUD 119K-161K

Agent Orchestration | Agent systems | Bias Testing | Context engineering | Evaluation datasets

Coaching | Equity | Extended end of year break | Home workspace setup budget | Inclusive parental leave

Senior-level Full Time

Melbourne

24d ago

[Job - 30085] Senior Data Scientist (LLM), Brazil BRL 126K-160K

A/B | A/B Testing | AWS | B testing | Cloud Computing

Childcare support | Dental insurance | Discount club | Extended parental leave | Gympass/Wellhub

Senior-level Full Time

Brazil

27d ago

Senior Applied AI/ML Scientist - Compass CAD 171K-235K

Agentic AI | Artificial Intelligence | Backend Development | Cost Quality Tradeoffs | Data Engineering

Comprehensive benefits | Hybrid work schedule | Remote work up to 4 weeks per year

Senior-level Full Time

Kitchener-Waterloo, ON; Toronto, ON R

27d ago

Senior Applied AI/ML Scientist - Compass USD 196K-269K

Agentic AI | Artificial Intelligence | Cost Optimization | Data Modeling | Data strategy

Hybrid work flexibility | Remote work option

Senior-level Full Time

New York City, NY; San Francisco, … R

27d ago

AI Engineer (6 months contract) AED 168K-180K

AI Search | AWS | Agentic Workflows | Anthropic API | Apache Spark

Mid-level Contract

Abu Dhabi, United Arab Emirates

27d ago

AI Engineering Lead, New York USD 200K-300K

Agent evaluation | Agent systems | Amazon Web Services | Azure | CI/CD

5 days per week work schedule | Flexible working hours | Supportive work environment

Senior-level Full Time

New York, New York, United States

28d ago

Senior AI Engineer USD 190K-250K

A/B | A/B Testing | Adversarial Testing | Airflow | B testing

401k contributions | CEU support | Continuing Medical Education Support | Fertility and family planning | Fitness perks

Senior-level Full Time

United States - Remote R

29d ago

Senior AI Engineer USD 125K-156K

API Design | CI/CD | Database migrations | Dataset Management | Document indexing

401k match | Annual bonus | Annual lunch and learn | Dental insurance | Disability insurance

Senior-level Full Time

Remote R

29d ago

AI Engineering Lead INR 3264K-5000K

AI Observability | Agent systems | Amazon Web Services | Azure | CI/CD

5 days a week working schedule | Collaborative team | Competitive compensation | Flexible working hours | Supportive work environment

Senior-level Full Time

Bengaluru, Karnataka, India; Gurugram, Haryana, India

29d ago

Solutions Architect INR 2200K-4500K

AWS CDK | AWS CloudFormation | AWS Lambda | Agent Orchestration | AgentCore)

Senior-level Full Time

Bengaluru, Karnataka, IND

1mo ago

Solutions Architect INR 2200K-4500K

AWS CDK | AWS IAM | AWS KMS | AWS Lambda | AWS Step Functions

Senior-level Full Time

Bengaluru, Karnataka, IND

1mo ago

Staff AI research scientist USD 234K-296K

Adversarial Training | Agentic Systems | Benchmark design | Data Curation | Data Generation

Company holidays | Company offsites | Dental insurance | Dependent FSA | Fertility support

Senior-level Full Time

San Francisco, CA

1mo ago

Senior Machine Learning Engineer USD 188K-282K

Adversarial Training | Calibration monitoring | Continuous batching | DPO | Deep learning

Senior-level Full Time

Palo Alto, CA

1mo ago

Principal Software Engineer, Applied AI USD 188K-258K

AWS | Azure | Cloud platform | Context engineering | Cost Optimization

401k matching | Commuting benefits | Disability insurance | Flexible PTO | Health savings account

Senior-level Full Time

Seattle

1mo ago

AI Engineer USD 115K-183K

API Integration | Amazon AgentCore | Amazon Bedrock | Amazon CloudWatch | CI/CD

Flexible time off | Health benefits | Learning and development | Nine Company-Wide Holidays | Remote work

Mid-level Full Time

St. Petersburg, Florida R

1mo ago

Senior AI Research Scientist EUR 61K-86K

Agent Orchestration | Benchmarking | Context Management | Context engineering | Deep learning

Relocation assistance

Senior-level Full Time

Madrid

1mo ago

Agentic AI Engineer INR 3000K-5000K

AWS | Access Control | Autogen | Azure | CI/CD

Senior-level Full Time

Bengaluru, Karnataka, India

1mo ago

Senior Applied AI Engineer USD 160K-210K

API Design | AWS | CI/CD | Circuit Breakers | Docker

Dynamic work environment | Flexible working hours

Senior-level Full Time

US - Remote, Canada - Remote R

1mo ago

Staff Applied AI Scientist AUD 148K-180K

Agentic Systems | Data Labelling | Evaluation datasets | Human-in-the-loop | LLM-as-judge

Coaching | Extended end of year break | Home workspace setup budget | Inclusive parental leave | Learning programs

Senior-level Full Time

Melbourne

1mo ago

Gen AI Engineer (M/F) - Internship - Paris EUR 16K-19K

Evaluation metrics | FastAPI | Flask | GitLab | Kubernetes

Remote work | Skills development | Team building

Entry-level Internship

Paris, France

1mo ago

Head of Data Quality USD 280K-350K

Automation | Cross-functional | Cross-functional leadership | Data Analysis | Data Governance

401k match | Dental insurance | Medical insurance | Vision insurance

Executive-level Full Time

Redwood City, CA (Hybrid) R

1mo ago

Predictive Science - AI Engineering & Prompt Architecture Lead - Vice President INR 2500K-4500K

A/B | A/B Testing | Attention | B testing | CI/CD

Senior-level Full Time

India

1mo ago

Full Stack AI Engineer Manager SGD 85K-138K

A/B | A/B Testing | AWS | AWS Lambda | AgentOps

Mid-level Full Time

Singapore, Raffles City Tower

1mo ago

Full Stack AI Engineer Ass Manager SGD 90K-133K

A/B | A/B Testing | AWS | AgentOps | Audit trails

Mid-level Full Time

Singapore, Raffles City Tower

1mo ago

Full Stack AI Engineer Senior Manager SGD 128K-213K

A2A Coordination | AWS | AgentOps | Agentic AI | Audit trails

Senior-level Full Time

Singapore, Raffles City Tower

1mo ago

Full Stack AI Engineer Associate Director SGD 85K-138K

AI architecture | AgentOps | Agentic AI | Cloud Native | Cloud-native development

Mid-level Full Time

Singapore, Raffles City Tower

1mo ago

Find jobs in AI/ML, Data Science and Big Data