Senior Software Engineer - AI Interaction Evaluator (Codex / Claude Code, up to $200/hr)
Tasks
- Assess response sense correctness alignment and engineering judgment
- Assess trust building guidance and output clarity
- Define response quality rubric and scoring levels
- Evaluate AI behavior in real world scenarios
- Evaluate AI generated coding interactions end to end
- Provide subjective but rigorous opinionated feedback
- Review quality of explanations and reasoning
Perks/Benefits
- N/A
Skills/Tech-stack
Claude Code | Cursor | JavaScript | OpenAI Codex | Python | TypeScript
Education
N/A
Roles
AI | AI Interaction Evaluator | Engineer | Evaluator | Software Engineer
Regions
Countries
Canada | Ghana | United States
States
Alberta, CA | British Columbia, CA | Manitoba, CA | Ontario, CA | Quebec, CA | District of Columbia, US | Florida, US | Georgia, US | Oklahoma, US | Tennessee, US | Texas, US | Illinois, US | Indiana, US | Ohio, US | Colorado, US | Nevada, US | Idaho, US | Washington, US | Massachusetts, US | Pennsylvania, US | Virginia, US
Cities
Chicago, Illinois, US | Toronto, Ontario, CA | Montréal, Quebec, CA | Philadelphia, Pennsylvania, US | San Antonio, Texas, US | Dallas, Texas, US | Calgary, Alberta, CA | Edmonton, Alberta, CA | Jacksonville, Florida, US | Austin, Texas, US | Fort Worth, Texas, US | Columbus, Ohio, US | Indianapolis, Indiana, US | Ottawa, Ontario, CA | Winnipeg, Manitoba, CA | Seattle, Washington, US | Denver, Colorado, US | Nashville, Tennessee, US | Oklahoma City, Oklahoma, US | Boston, Massachusetts, US | Mississauga, Ontario, CA | Las Vegas, Nevada, US | Vancouver, British Columbia, CA | Atlanta, Georgia, US | Virginia Beach, Virginia, US | Miami, Florida, US | Arlington, Texas, US | Reno, Nevada, US | Chesapeake, Virginia, US | Richmond, Virginia, US | Arlington, Virginia, US | Savannah, Georgia, US | Boise, Idaho, US | Fairfax, Virginia, US
Related jobs
-
AWS | Access Control | Alerting | Audit Logging | AzureCollaborative environment | Fully remote option in Canada | Mentoring and leadership opportunities | Opportunity to shape organization wide machine learning and generative AI infrastructure strategySenior-level Full TimeCanada R8h ago
-
AWS | AWS CloudFormation | Alerting | Amazon Kinesis | Apache FlinkCollaborative environment | Comprehensive medical benefits | Financial benefits | Remote work (Canada) | Technical ownershipMid-level Full TimeCanada R8h ago
-
Senior Manager, AI Engineering USD 150K-187KAI Act | AI RMF | Agent Frameworks | Bias Testing | Deep learning401k match | EAP Coverage | Employee stock purchase plan | Hybrid work model | Medical/Dental/Vision insuranceSenior-level Full TimePrinceton, New Jersey, United States; San … R8h ago
-
Claude Code | Cursor | JavaScript | OpenAI Codex | PythonMid-level Full TimeMiami R10h ago
-
Staff Machine Learning Engineer, Traffic Intelligence USD 212K-265KBayesian Calibration | Bayesian Inference | Calibration | Causal Inference | Data PipelinesSenior-level Full TimeUnited States R13h ago
-
Senior Machine Learning Engineer USD 150K-241KC++ | Cloud Architecture | Computer Vision | Data Privacy | EncryptionDiscretionary paid time off | Emotional & mental wellness support | Employer 401k match | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R13h ago
-
Active Learning | Auto-labeling | Computer Vision | Confidence Scores | Data BalancingBonus program | Company benefits program | Equity incentive planSenior-level Full TimeMountain View, CA, USA; San Francisco, … R13h ago
-
Solution Engineer USD 165K-216KCloud Computing | Data Lake | Data Warehouse | Data fabric | Data meshTravelSenior-level Full TimeUS-OH-Remote R15h ago
-
AI Solutions Intern (4-Month Contract) CAD 56K-56KAI Studio | Azure | Azure AI | Azure AI Studio | Azure DevOpsEntry-level Contract Full Time InternshipVancouver, British Columbia, Canada R15h ago
-
Principal Data Engineer (Remote) USD 151K-189KAmazon Kinesis | Apache Spark | CI/CD | Cloud Data | Cloud Data Services12 weeks new parent leave | Coworking support | Flexible PTO | Learning & development stipend | Time allotment for learning and developmentSenior-level Full TimeUnited States R15h ago
-
Lead Data Engineer (Remote) USD 126K-157KAWS | AWS Redshift | Amazon Kinesis | Azure | Azure DataCoworking support | Flexible PTO | Learning and development stipend | New parent leave | Remote workSenior-level Full TimeUnited States R16h ago
-
Senior Machine Learning Engineer, Vehicle Perception USD 140K-230KAblation Studies | C plus plus | Cloud Computing | Computer Vision | Data CurationComprehensive benefits package | Hybrid work schedule | Incentive programsSenior-level Full TimePalo Alto, CA R16h ago
-
Senior Analytics Engineer USD 141K-176KAWS Glue | Amazon Athena | Apache Hive | Apache Spark | CI/CDGenerous time off | Healthcare | Occasional travel for in person meetings | Paid personal time off | Paid sick timeSenior-level Full TimeRemote - US R16h ago
-
Staff AI Scientist USD 170K-225KAdversarial Training | BERT | Data Generation | Data labeling | Dataset development401k plan | Annual events | CalTrain pass | Fastrak reimbursement | Fitness reimbursementSenior-level Full TimePalo Alto R16h ago
-
Senior Software Engineer (Forward Deployed) USD 180K-264KAWS | Automation | Data Contracts | Data Governance | Data QualityHybrid work | On-call support | Travel opportunitiesSenior-level Full TimeLondon - Hybrid; New York - … R17h ago
-
Data Scientist USD 90K-120KAWS | Agentic AI | Azure | Cypress | Data Analysis401k employer contribution | Remote work optionMid-level Full TimeRemote, USA R18h ago
-
Senior AI Engineer USD 170K-190KAPI Gateway | AWS Lambda | Amazon API | Amazon API Gateway | Amazon AuroraCareer advancement | Employee assistance program | Employee recognition | LinkedIn Learning Platform | Open PTOSenior-level Full TimeRemote US R19h ago
-
Senior Machine Learning Engineer I, AI & ML Platform USD 183K-205KAWS | Azure | DevOps | K8s | Kubernetes401k match | Child care backup care and elder care | Donation matching | Fertility care support | Fitness Mindfulness Nutrition and Sleep SubscriptionSenior-level Full TimeSan Francisco, CA (Hybrid) R19h ago
-
AI Engineer (Remote) USD 169K-230KAWS | Agent systems | Application development | Automation | Azure401k plan | Accident insurance | Dental insurance | Disability insurance | Flexible spending accountExecutive-level Full TimeFort Washington, PA, United States R19h ago
-
Software Engineer, Storage USD 80K-105KAnsible | Ceph | Continuous Deployment | Continuous integration | Distributed Systems401k match | Anniversary bonus | Dental insurance premium paid by employer | Gym membership reimbursement | Health Insurance Premium Paid by EmployerMid-level Full TimeRemote - United States R19h ago
-
Senior Storage Engineer USD 130K-150KAutomation | Bash | Cause analysis | Ceph | Cloud infrastructure401k match | Company-paid health insurance | Gym membership reimbursement | Internet reimbursement | Paid sabbaticalSenior-level Full TimeRemote - United States R19h ago
-
Fullstack Software Engineer CAD 176K-226KAPI Design | Java | JavaScript | Node.js | Performance EngineeringSenior-level Full TimeCA-Ontario-Toronto R19h ago
-
Storage Operations Engineer USD 80K-100KAnsible | Automation | Bash | Block Storage | Ceph100 percent company paid insurance premiums | 401k match | Company paid Wellable subscription | Gym membership reimbursement | Internet reimbursementMid-level Full TimeRemote - United States R19h ago
-
Staff/Senior Agentic AI Engineers (Multiple roles) USD 180K-215KAPI Design | AWS | Agent Orchestration | Agentic AI | Cloud infrastructureSenior-level Full TimeSan Francisco, California R20h ago
-
APIs | AWS | Access Management | Azure | Azure DataFull-time role | Occasional travel within Alberta | Remote within CanadaSenior-level Full TimeEdmonton, Alberta; Calgary - Remote Position, … R20h ago