Senior AI Engineer, Agentic Evaluation & V&V

Remote R

USD 150K-250K Senior-level Full Time

@ S...

Apply Save

Found 17h ago

Tasks

Analyze system behavior and failure modes
Build evaluation frameworks for agentic AI systems
Create reusable SDK interfaces and adapters
Define and apply evaluation metrics
Design benchmark scenarios and scoring logic
Develop simulation based validation systems
Ensure testing documentation reproducibility
Implement experiment harnesses
Translate mission domain concepts into evaluation scenarios

Perks/Benefits

Skills/Tech-stack

Education

Bachelor of Engineering | Bachelor of Science | Master of Science

Roles

AI | AI Engineer | Engineer | Senior AI Engineer

Apply Save

Language: en | Views: 0 | Clicks: 0 | Saves: 0

Related jobs

Featured Feat. AI Engineer (MTS) USD 160K-300K

API Development | AWS | Amazon Web Services | Deep learning | FastAPI

Mentoring | Open source contributions | Remote work

Mid-level

Remote R

10h ago
Featured Feat. Data Engineer USD 80K-150K

Data Monitoring | Data Quality | Data Validation | ELT | ETL

Remote work

Entry-level

Remote R

10h ago
Principal Data Engineer USD 200K-240K

AWS | Agentic Workflows | Anomaly Detection | Batch pipelines | CCPA

401k plan | Commuter benefits | Flexible vacation | Life insurance | Long-term disability

Senior-level Full Time

Boulder, Colorado or New York City, … R

6h ago
AI Solution Strategist AUD 130K-180K

AI Agent | AI agent design | Agent Design | Artificial Intelligence | Conversational Design

Mid-level Full Time

Australia - Remote R

7h ago
Senior Data Scientist / AI Engineer - Generative AI SEK 686K-838K

Agentic patterns | Evaluation | Experimentation | Generative AI | LLM APIs

Diversity and inclusion initiatives | Hybrid working model | Work in English speaking environment

Senior-level Full Time

Stockholm, SE, 111 44 R

8h ago
Conseiller(ère) Stratégies analytiques - Développeur BI CAD 73K-122K

Azure DevOps | CI/CD | DAX | Data Governance | Data Ingestion

Flexible schedule | Four weeks vacation | Health insurance | Hybrid work options | Paid leave options

Mid-level Full Time

Montréal, QC, CA R

8h ago
Senior Principal Engineer - Data Engineering INR 3000K-4000K

AWS | Apache Spark | Automation | Azure Data | Azure Data Factory

Hybrid work environment | Inclusive culture | Professional development opportunities | Supportive leaders

Senior-level Full Time

Pune - Panchshil, India R

10h ago
Forward Deployed Machine Learning Engineer USD 165K-200K

Cloud Computing | Deep learning | Experimentation | Language Models | Language Processing

Senior-level Full Time

Remote R

12h ago
(Contract) Data Platform Engineer (AWS & Data Pipelines) EUR 65K-65K

Amazon ECS | Amazon EMR | Amazon Redshift | Apache Airflow | Apache Spark

Distributed team collaboration | Remote work

Mid-level Contract

Argentina (Remote); Brazil (Remote); Colombia (Remote); … R

13h ago
Senior Applied Scientist, Scheduling and Optimization CAD 130K-170K

API Development | Asynchronous programming | CP-SAT | CPLEX | Constraint Programming

401k retirement plan | Dental insurance | Healthcare | PTO | Remote work

Senior-level Full Time

Canada (Remote) R

13h ago
Senior Machine Learning Engineer USD 200K-230K

Batching | Cloud Inference | Computer Vision | Deep learning | Edge Computing

Dental insurance | Flexible PTO | Health insurance | Remote work | Vision insurance

Senior-level Full Time

Remote, US or Canada - NYC … R

13h ago
Senior Data Engineer USD 150K-165K

APIs | AWS | Automation | CI/CD | Data Pipelines

401k matching | Birthday day off | Fitness stipend | Floating holidays | Health benefits

Senior-level Full Time

United States R

13h ago
Senior Embedded Systems Engineer USD 170K-226K

ACAP | Ansible | Bash | Cellular networking | DNS

401k match | Dental insurance | Medical insurance | PTO | Sick days without limit

Senior-level Full Time

Chicago / Remote R

14h ago
Senior Data Engineer, Data Foundations & AI Platform USD 153K-207K

APIs | Alerting | Apache Spark | CI/CD | Data Lineage

401k with company match | Disability insurance | Flexible time off | Health, dental, and vision insurance | Leave of absence

Senior-level Full Time

United States (Remote) R

14h ago
AI/ML Engineering Manager

AI Governance | AWQ | AWS CDK | AWS SageMaker | Agent systems

Equipment and office stipend | Flexible PTO | Learning and development stipend | Medical insurance | Paid exams and certifications

Mid-level Full Time

MEXICO R

15h ago
AI/ML Engineering Manager USD 140K-185K

AWS | AWS CDK | AWS CloudFormation | AWS Glue | Airflow

100 percent remote work | Equipment and office stipend | Flexible PTO | Generous holidays | Individual professional development plan

Mid-level Full Time

ARGENTINA R

15h ago
AI/ML Engineering Manager

AWS | AWS Glue | Airflow | Amazon Bedrock | Amazon Kinesis

100% remote work | Annual learning stipend | Equipment stipend | Flexible time off | Office stipend

Mid-level Full Time

BRAZIL R

15h ago
AI/ML Engineering Manager USD 140K-215K

AWS | AWS CDK | AWS CloudFormation | AWS Glue | Agent systems

401k plan | Company laptop | Dental insurance | Equipment and office stipend | Flexible spending account

Mid-level Full Time

USA R

15h ago
Applied AI Engineer, Agentic Systems INR 2755K-5500K

.NET | API Development | Agent Orchestration | Anthropic API | C Sharp

Senior-level Full Time

Remote - India R

15h ago
Machine Learning Lead (LLM) USD 165K-210K

Agentic Workflows | Data Analysis | Deep learning | Docker | Embeddings

Dental coverage | Health coverage | Medical coverage | Meetups in NYC and DC | Remote-first

Senior-level Full Time

Remote R

16h ago
Product Manager, AI Solutions CNY 304K-399K

Analytics | Backlog Management | Canva | Data Analysis | Demos

Direct Access To Founding Team | Health insurance | High autonomy | Low bureaucracy | Optional hack weeks

Mid-level Full Time

Shenzhen R

16h ago
Senior Data Engineer USD 160K-190K

A/B | A/B Testing | B testing | Clickstream | DBT

401k match | Health and welfare benefits | Hybrid work | Paid Holidays | Paid vacation

Senior-level Full Time

United States R

16h ago
Sr. Data Engineer II (6516) USD 152K-188K

AWS | Apache NiFi | Cloudera | Data Architecture | Data Preparation

401k matching | Dental insurance | Dependent care | Employee Assistance and Wellness Programs | Flexible work arrangements

Mid-level Full Time

Remote R

16h ago
AI Engineer - Model Performance USD 165K-250K

Attention Backend | Audio Processing | Batching | CUDA | CUDA graph

Async communication | Innovation-focused culture | Remote work | Startup environment | Supportive team

Mid-level Full Time

SF Hybrid R

16h ago
Senior Data Engineer USD 95K-135K

AWS | Airflow | C++ | Cassandra | Cloud platform

401k matching | Community service days | Dental insurance | Disability benefits | Fertility and adoption benefits

Senior-level Full Time

Chicago, IL R

17h ago