AI Engineer, Quality
San Francisco, CA or Remote (USA)
R
USD 170K-220K Senior-level Full Time Found 10d ago
Tasks
- Advocate for evaluation-driven development
- Build automated model evaluation pipelines
- Create evaluation datasets with SMEs
- Define evaluation standards and best practices
- Design and build unified evaluation platform
- Design evaluation harnesses and comparison frameworks
- Design prompts and orchestration systems
- Develop observability systems for production
- Implement quality guardrails and monitoring
- Integrate LLMs, tools, retrieval systems
- Own evaluation infrastructure stack integration
- Own large product areas related to quality and evaluation
- Partner with product and ML engineers on evaluation requirements
- Translate customer problems into agent behaviors
- Use AI for design, testing, and iteration
Perks/Benefits
- N/A
Skills/Tech-stack
AI-native | AI-native engineering | Agent Orchestration | Automation | Data-Driven Decision Making | Data-driven | Decision Making | Embedding Models | Evaluation Frameworks | LLMs | Machine Learning | Monitoring Systems | Observability | Postgres | Prompt engineering | Python | RAG Architectures | React | Tracing infrastructure | TypeScript | Vector Databases
Education
N/A
Roles
Regions
Countries
States
Language: en |
Views: 0 |
Clicks: 0
Related jobs
-
Senior AI Engineer – Data and AI USD 130K-180KAI infrastructure | AI model governance | API frameworks | CD pipelines | CI/CDFamily support programs | Hybrid work opportunity | Medical, dental & vision coverage | Paid Holidays | Paid time offSenior-level Full TimePlano, TX, United States R3d ago
-
Senior AI Engineer – Data and AI USD 125K-165KAI model governance | API frameworks | CI/CD | Cloud Native | Cloud-native development401k match | Family support | Hybrid work option | Medical/Dental/Vision | Paid HolidaysSenior-level Full TimeCamas, WA, United States R3d ago
-
Anomaly Detection | Automation frameworks | Clustering | Data Analysis | Distributed SystemsEntry-level Full Time6314 Remote/Teleworker US, United States R3d ago
-
Senior-level Full TimeRemote, VA, US R3d ago
-
AI | APIs | Angular | Automation | C++Flexible remote | Impactful projects | Professional development | Team cultureSenior-level Full TimeIowa R5d ago
-
Principal Engineer - AI USD 186K-286KAI Platform | AI Platform Architecture | Agent-based | Agent-based Architectures | Anomaly DetectionFinancial benefits | Medical benefits | Paid time off | Parental leave | Stock optionsSenior-level Full TimeRemote Oregon, United States R6d ago
-
Sr. Applied AI Engineer USD 150K-225KAPI Design | Agent Frameworks | Containerization | Deployment | DockerFlexible schedule | Growth opportunities | Impactful work | Remote work | Retirement matchSenior-level Full TimeUnited States R8d ago
-
AI Engineer USD 100K-140KArtificialIntelligence | Collaboration | Communication | DataScience | Databricks401k | Dental | Medical | PaidHolidays | PaidVacationsEntry-level Full TimeRemote - Michigan, United States R9d ago
-
AI Engineer USD 109K-156KAPI Integration | AWS | Algorithms | Artificial Intelligence | Azure401k match | AD&D | Dental | LTD | Life insuranceMid-level Full TimeRemote, United States R9d ago
-
AI Engineer USD 111K-176KArtificialIntelligence | Communication | DataScience | Databricks | Domo401k | Dental | Medical | PaidHolidays | PaidVacationsEntry-level Full TimeRemote - Michigan, United States R10d ago
-
Forward Deployed Engineer (AI/Agents) USD 165K-242KAPI Development | Agent systems | Algorithms | Cloud Platforms | Cloud Platforms (AWSInsurance coverage | Paid time off | Parental leave | Recognition rewardsSenior-level Full TimeUnited States - Remote R12d ago