Senior Engineer - AI Interaction Evaluator - US and Canada only
Tasks
- Assess alignment with strong engineering judgment
- Assess quality of explanations and reasoning
- Define criteria for high quality interactions
- Evaluate AI generated coding interactions end to end
- Evaluate trust building and guidance quality
- Judge response usefulness and correctness at high level
- Provide opinionated feedback on what worked and what didnt
Perks/Benefits
Skills/Tech-stack
AI Assisted Development | Claude Code | Cursor | JavaScript | OpenAI Codex | Prompt evaluation | Python | TypeScript
Education
N/A
Regions
Countries
Canada | Ghana | United States
States
Alberta, CA | British Columbia, CA | Manitoba, CA | Ontario, CA | Quebec, CA | District of Columbia, US | Florida, US | Georgia, US | Oklahoma, US | Tennessee, US | Texas, US | Illinois, US | Indiana, US | Ohio, US | Colorado, US | Nevada, US | Idaho, US | Washington, US | Massachusetts, US | Pennsylvania, US | Virginia, US
Cities
Chicago, Illinois, US | Toronto, Ontario, CA | Montréal, Quebec, CA | Philadelphia, Pennsylvania, US | San Antonio, Texas, US | Dallas, Texas, US | Calgary, Alberta, CA | Edmonton, Alberta, CA | Jacksonville, Florida, US | Austin, Texas, US | Fort Worth, Texas, US | Columbus, Ohio, US | Indianapolis, Indiana, US | Ottawa, Ontario, CA | Winnipeg, Manitoba, CA | Seattle, Washington, US | Denver, Colorado, US | Nashville, Tennessee, US | Oklahoma City, Oklahoma, US | Boston, Massachusetts, US | Mississauga, Ontario, CA | Las Vegas, Nevada, US | Vancouver, British Columbia, CA | Atlanta, Georgia, US | Virginia Beach, Virginia, US | Miami, Florida, US | Arlington, Texas, US | Reno, Nevada, US | Chesapeake, Virginia, US | Richmond, Virginia, US | Arlington, Virginia, US | Savannah, Georgia, US | Boise, Idaho, US | Fairfax, Virginia, US
Related jobs
-
AWS | Access Control | Alerting | Audit Logging | AzureCollaborative environment | Fully remote option in Canada | Mentoring and leadership opportunities | Opportunity to shape organization wide machine learning and generative AI infrastructure strategySenior-level Full TimeCanada R9h ago
-
AWS | AWS CloudFormation | Alerting | Amazon Kinesis | Apache FlinkCollaborative environment | Comprehensive medical benefits | Financial benefits | Remote work (Canada) | Technical ownershipMid-level Full TimeCanada R9h ago
-
AI Assisted Development | Analytical Databases | ClickHouse | Data Engineering | Data PipelinesFully remote work | Mentorship opportunities | Significant autonomy | Technical leadership autonomySenior-level Full TimeCanada R10h ago
-
AI Assisted Development | Backup and Recovery | ClickHouse | Cloud Compliance | Data PipelinesAsynchronous collaboration | Fully remote work | High-autonomy culture | Opportunities for experimentation and continuous learning | Ownership based engineeringSenior-level Full TimeCanada R10h ago
-
Senior Software Engineer - AI Interaction Evaluator (Codex / Claude Code, up to $200/hr) USD 200K-400KClaude Code | Cursor | JavaScript | OpenAI Codex | PythonSenior-level Full TimeMiami R12h ago
-
AI Security Engineer USD 140K-200K800-53 | ABAC | AI RMF | AI Studio | AKS401k matching | Dental insurance | Employee development program | Employee referral program | Medical insuranceSenior-level Full TimeRemote, MD, US R13h ago
-
Junior AI Engineer USD 86K-120KAPI Integration | Authentication | Cloud deployment | Docker | Error Handling401k matching | BYOD reimbursement | Company holidays | Flexible spending accounts | Health savings accountEntry-level Full TimeHouston, Texas, United States (Hybrid) R14h ago
-
Staff Machine Learning Engineer, Traffic Intelligence USD 212K-265KBayesian Calibration | Bayesian Inference | Calibration | Causal Inference | Data PipelinesSenior-level Full TimeUnited States R15h ago
-
Senior Machine Learning Engineer USD 150K-241KC++ | Cloud Architecture | Computer Vision | Data Privacy | EncryptionDiscretionary paid time off | Emotional & mental wellness support | Employer 401k match | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R15h ago
-
Active Learning | Auto-labeling | Computer Vision | Confidence Scores | Data BalancingBonus program | Company benefits program | Equity incentive planSenior-level Full TimeMountain View, CA, USA; San Francisco, … R15h ago
-
Solution Engineer USD 165K-216KCloud Computing | Data Lake | Data Warehouse | Data fabric | Data meshTravelSenior-level Full TimeUS-OH-Remote R17h ago
-
Principal Data Engineer (Remote) USD 151K-189KAmazon Kinesis | Apache Spark | CI/CD | Cloud Data | Cloud Data Services12 weeks new parent leave | Coworking support | Flexible PTO | Learning & development stipend | Time allotment for learning and developmentSenior-level Full TimeUnited States R17h ago
-
Lead Data Engineer (Remote) USD 126K-157KAWS | AWS Redshift | Amazon Kinesis | Azure | Azure DataCoworking support | Flexible PTO | Learning and development stipend | New parent leave | Remote workSenior-level Full TimeUnited States R17h ago
-
Senior Machine Learning Engineer, Vehicle Perception USD 140K-230KAblation Studies | C plus plus | Cloud Computing | Computer Vision | Data CurationComprehensive benefits package | Hybrid work schedule | Incentive programsSenior-level Full TimePalo Alto, CA R17h ago
-
Senior Analytics Engineer USD 141K-176KAWS Glue | Amazon Athena | Apache Hive | Apache Spark | CI/CDGenerous time off | Healthcare | Occasional travel for in person meetings | Paid personal time off | Paid sick timeSenior-level Full TimeRemote - US R18h ago
-
Senior Software Engineer (Forward Deployed) USD 180K-264KAWS | Automation | Data Contracts | Data Governance | Data QualityHybrid work | On-call support | Travel opportunitiesSenior-level Full TimeLondon - Hybrid; New York - … R19h ago
-
Data Scientist USD 90K-120KAWS | Agentic AI | Azure | Cypress | Data Analysis401k employer contribution | Remote work optionMid-level Full TimeRemote, USA R20h ago
-
Senior AI Engineer USD 170K-190KAPI Gateway | AWS Lambda | Amazon API | Amazon API Gateway | Amazon AuroraCareer advancement | Employee assistance program | Employee recognition | LinkedIn Learning Platform | Open PTOSenior-level Full TimeRemote US R20h ago
-
Senior Machine Learning Engineer I, AI & ML Platform USD 183K-205KAWS | Azure | DevOps | K8s | Kubernetes401k match | Child care backup care and elder care | Donation matching | Fertility care support | Fitness Mindfulness Nutrition and Sleep SubscriptionSenior-level Full TimeSan Francisco, CA (Hybrid) R20h ago
-
AI Engineer (Remote) USD 169K-230KAWS | Agent systems | Application development | Automation | Azure401k plan | Accident insurance | Dental insurance | Disability insurance | Flexible spending accountExecutive-level Full TimeFort Washington, PA, United States R21h ago
-
Software Engineer, Storage USD 80K-105KAnsible | Ceph | Continuous Deployment | Continuous integration | Distributed Systems401k match | Anniversary bonus | Dental insurance premium paid by employer | Gym membership reimbursement | Health Insurance Premium Paid by EmployerMid-level Full TimeRemote - United States R21h ago
-
Senior Storage Engineer USD 130K-150KAutomation | Bash | Cause analysis | Ceph | Cloud infrastructure401k match | Company-paid health insurance | Gym membership reimbursement | Internet reimbursement | Paid sabbaticalSenior-level Full TimeRemote - United States R21h ago
-
Fullstack Software Engineer CAD 176K-226KAPI Design | Java | JavaScript | Node.js | Performance EngineeringSenior-level Full TimeCA-Ontario-Toronto R21h ago
-
Storage Operations Engineer USD 80K-100KAnsible | Automation | Bash | Block Storage | Ceph100 percent company paid insurance premiums | 401k match | Company paid Wellable subscription | Gym membership reimbursement | Internet reimbursementMid-level Full TimeRemote - United States R21h ago
-
Staff/Senior Agentic AI Engineers (Multiple roles) USD 180K-215KAPI Design | AWS | Agent Orchestration | Agentic AI | Cloud infrastructureSenior-level Full TimeSan Francisco, California R21h ago