Senior AI Engineer, Agentic Evaluation & V&V

Remote R

USD 150K-250K Senior-level Full Time

@ S...

Apply Save

Found 16h ago

Tasks

Analyze system behavior and failure modes
Build evaluation frameworks for agentic AI systems
Create reusable SDK interfaces and adapters
Define and apply evaluation metrics
Design benchmark scenarios and scoring logic
Develop simulation based validation systems
Ensure testing documentation reproducibility
Implement experiment harnesses
Translate mission domain concepts into evaluation scenarios

Perks/Benefits

Skills/Tech-stack

Education

Bachelor of Engineering | Bachelor of Science | Master of Science

Roles

AI | AI Engineer | Engineer | Senior AI Engineer

Apply Save

Language: en | Views: 0 | Clicks: 0 | Saves: 0

Related jobs

Featured Feat. AI Engineer (MTS) USD 160K-300K

API Development | AWS | Amazon Web Services | Deep learning | FastAPI

Mentoring | Open source contributions | Remote work

Mid-level

Remote R

9h ago
Featured Feat. Data Engineer USD 80K-150K

Data Monitoring | Data Quality | Data Validation | ELT | ETL

Remote work

Entry-level

Remote R

9h ago
Principal Data Engineer USD 200K-240K

AWS | Agentic Workflows | Anomaly Detection | Batch pipelines | CCPA

401k plan | Commuter benefits | Flexible vacation | Life insurance | Long-term disability

Senior-level Full Time

Boulder, Colorado or New York City, … R

5h ago
AI Solution Strategist AUD 130K-180K

AI Agent | AI agent design | Agent Design | Artificial Intelligence | Conversational Design

Mid-level Full Time

Australia - Remote R

6h ago
Senior Data Scientist / AI Engineer - Generative AI SEK 686K-838K

Agentic patterns | Evaluation | Experimentation | Generative AI | LLM APIs

Diversity and inclusion initiatives | Hybrid working model | Work in English speaking environment

Senior-level Full Time

Stockholm, SE, 111 44 R

7h ago
Senior Applied Scientist, Scheduling and Optimization CAD 130K-170K

API Development | Asynchronous programming | CP-SAT | CPLEX | Constraint Programming

401k retirement plan | Dental insurance | Healthcare | PTO | Remote work

Senior-level Full Time

Canada (Remote) R

12h ago
Senior Machine Learning Engineer USD 200K-230K

Batching | Cloud Inference | Computer Vision | Deep learning | Edge Computing

Dental insurance | Flexible PTO | Health insurance | Remote work | Vision insurance

Senior-level Full Time

Remote, US or Canada - NYC … R

12h ago
Senior Data Engineer USD 150K-165K

APIs | AWS | Automation | CI/CD | Data Pipelines

401k matching | Birthday day off | Fitness stipend | Floating holidays | Health benefits

Senior-level Full Time

United States R

12h ago
Senior Embedded Systems Engineer USD 170K-226K

ACAP | Ansible | Bash | Cellular networking | DNS

401k match | Dental insurance | Medical insurance | PTO | Sick days without limit

Senior-level Full Time

Chicago / Remote R

13h ago
Senior Data Engineer, Data Foundations & AI Platform USD 153K-207K

APIs | Alerting | Apache Spark | CI/CD | Data Lineage

401k with company match | Disability insurance | Flexible time off | Health, dental, and vision insurance | Leave of absence

Senior-level Full Time

United States (Remote) R

13h ago
AI/ML Engineering Manager

AI Governance | AWQ | AWS CDK | AWS SageMaker | Agent systems

Equipment and office stipend | Flexible PTO | Learning and development stipend | Medical insurance | Paid exams and certifications

Mid-level Full Time

MEXICO R

14h ago
AI/ML Engineering Manager USD 140K-185K

AWS | AWS CDK | AWS CloudFormation | AWS Glue | Airflow

100 percent remote work | Equipment and office stipend | Flexible PTO | Generous holidays | Individual professional development plan

Mid-level Full Time

ARGENTINA R

14h ago
AI/ML Engineering Manager

AWS | AWS Glue | Airflow | Amazon Bedrock | Amazon Kinesis

100% remote work | Annual learning stipend | Equipment stipend | Flexible time off | Office stipend

Mid-level Full Time

BRAZIL R

14h ago
AI/ML Engineering Manager USD 140K-215K

AWS | AWS CDK | AWS CloudFormation | AWS Glue | Agent systems

401k plan | Company laptop | Dental insurance | Equipment and office stipend | Flexible spending account

Mid-level Full Time

USA R

14h ago
Applied AI Engineer, Agentic Systems INR 2755K-5500K

.NET | API Development | Agent Orchestration | Anthropic API | C Sharp

Senior-level Full Time

Remote - India R

14h ago
Machine Learning Lead (LLM) USD 165K-210K

Agentic Workflows | Data Analysis | Deep learning | Docker | Embeddings

Dental coverage | Health coverage | Medical coverage | Meetups in NYC and DC | Remote-first

Senior-level Full Time

Remote R

15h ago
Product Manager, AI Solutions CNY 304K-399K

Analytics | Backlog Management | Canva | Data Analysis | Demos

Direct Access To Founding Team | Health insurance | High autonomy | Low bureaucracy | Optional hack weeks

Mid-level Full Time

Shenzhen R

15h ago
Sr. Data Engineer II (6516) USD 152K-188K

AWS | Apache NiFi | Cloudera | Data Architecture | Data Preparation

401k matching | Dental insurance | Dependent care | Employee Assistance and Wellness Programs | Flexible work arrangements

Mid-level Full Time

Remote R

15h ago
AI Engineer - Model Performance USD 165K-250K

Attention Backend | Audio Processing | Batching | CUDA | CUDA graph

Async communication | Innovation-focused culture | Remote work | Startup environment | Supportive team

Mid-level Full Time

SF Hybrid R

15h ago
Senior Data Engineer USD 95K-135K

AWS | Airflow | C++ | Cassandra | Cloud platform

401k matching | Community service days | Dental insurance | Disability benefits | Fertility and adoption benefits

Senior-level Full Time

Chicago, IL R

16h ago
Senior Data Engineer USD 95K-135K

AWS | Airflow | C++ | Cassandra | Cloud platform

401k matching | Community service days | Dental insurance | Disability benefits | Fertility and adoption benefits

Senior-level Full Time

Denver, CO R

16h ago
Senior Data Engineer USD 137K-170K

AWS | Airflow | Apache Spark | Azure | C++

401k matching | Community service days | Dental insurance | Disability benefits | Fertility and adoption benefits

Senior-level Full Time

Houston, TX R

16h ago
Senior Data Engineer USD 137K-170K

AWS | Airflow | Apache Spark | C plus plus | Cassandra

401k matching | Community service days | Dental insurance | Disability benefits | Fertility and adoption benefits

Senior-level Full Time

Dallas, TX R

16h ago
Staff Software Engineer - Semantic Layer INR 3000K-4000K

API Design | Amazon Redshift | Backwards Compatibility | BigQuery | Compilation

Senior-level Full Time

India - Remote R

17h ago
AI Research Engineer GBP 120K-200K

CUDA | Data Analysis | Deep learning | Machine Learning | PyTorch

Async communication | Autonomy | Meeting-free days | Remote work

Mid-level Full Time

Hybrid (UK) R

17h ago