AI Quality Engineer
Tasks
- Build evaluation metrics and accuracy reports
- Design test cases and regression suites
- Diagnose extraction failures
- Document failure modes with examples
- Feed findings into validation logic
- Identify schema gaps and ambiguities
- Improve RAG with retrieval diagnosis
- Improve system instructions and context design
- Iterate on prompt engineering
- Maintain gold standard document sets
- Run daily accuracy evaluations
Perks/Benefits
- N/A
Skills/Tech-stack
Data Validation | Evaluation metrics | JSON Schema | LLM API | Language Models | Large Language Models | Prompt engineering | Python | RAG evaluation | Retrieval-Augmented Generation
Education
N/A
Related jobs
-
Computer Vision AI & ML Engineer USD 100K-300K3D Geometry | 3D Scene | 3D Scene Understanding | Computer Vision | Data AugmentationEntry-level Full TimeSan Mateo, CA4h ago
-
AI Engineer at Lumion USD 148K-200KAPI Design | Data Modeling | Debugging | Guardrails | Language Models401k | Health dental vision reimbursement | MacBook Pro | Relocation support | Tech stipendMid-level Full TimeSouth Jordan, UT, US, CA, US7h ago
-
AI Research Scientist- GenAI USD 165K-185KC++ | Deep learning | Foundation Models | Inference Optimization | Language ModelsMid-level Full TimePittsburgh, PA, United States8h ago
-
Senior Software Engineer USD 107K-150KAWS | Agile | Datadog | GitHub | GitHub ActionsOn-call rotation | Remote workSenior-level Full TimeCosta Mesa, CA, United States R12h ago
-
Senior-level Full TimeChantilly, VA12h ago
-
AI Engineer USD 109K-140KAgent Orchestration | Computer Vision | Data Validation | Data extraction | Document ClassificationMid-level Full TimeMorristown, NJ, United States13h ago
-
Summer 2026 Data Engineer USD 41K-50KAPIs | Agile | Azure Data | Azure Data Factory | Azure Data LakeExposure to real-world projects | Learning and development opportunities | MentorshipEntry-level InternshipBoston, MA, United States13h ago
-
Early-Career Network Engineer (RAN Optimization) USD 85K-130K4G | 5G | Automation | C Band | CBRS401k match | Dental insurance | Disability insurance | Educational assistance | Financial wellness programsMid-level Full TimePlano,Texas,United States R13h ago
-
Senior Embedded Software Engineer USD 146K-196KARM Cortex | C# | Digital Signal | Digital Signal Processing | Embedded Linux401k match | Dental insurance | Employee assistance program | FSA | Flexible scheduleSenior-level Full TimeCamarillo, CA, United States13h ago
-
Data Engineer USD 126K-208KAPI Integration | Airflow | Amazon Web Services | BigQuery | CCPADEI initiatives | Dental benefits | Employee rewards program | Medical benefits | Mental health supportMid-level Full TimeRemote, United States R13h ago
-
Alerting | Ansible | Bash | CI/CD | CephRemote workSenior-level Full TimeUnited States, United States R14h ago
-
Ansible | Bash | CI/CD | CentOS | CephContract-to-hire | No sponsorship | Remote workSenior-level Full TimeUnited States, United States R14h ago
-
Associate Consultant, Generative AI USD 80K-110KAWS | Azure | CI/CD | Chroma | Docker401k matching | Commuter benefits | Dental insurance | Employee assistance program | Flexible spending accountMid-level Full TimeNew York, NY, United States15h ago
-
HPG Big Data Engineer / Senior-Level USD 119K-164KAgile | Azure Data | Azure Data Lake | Azure Data Lake Storage | Azure FunctionsSenior-level Full TimeNashville, TN, United States15h ago
-
kdb+ Contractor (Capital Markets) USD 150K-200KAWS | Azure | Electronic Trading | GCP | Kdb PlusContract opportunities | On site work 5 days per week | Permanent opportunitiesSenior-level ContractNew York, NY, United States15h ago
-
Senior kdb+ Engineer USD 150K-200KAI Tooling | AWS | Azure | Capital Markets | Electronic TradingFlexible holiday allowance | Healthcare and wellbeing | Hybrid working | Leadership development | Paid certificationsSenior-level Full TimeNew York, NY, United States16h ago
-
Platform and Integrations Engineer USD 100K-200KAnthropic | GraphQL | Next.js | Node.js | OpenAIIn-person work | Medical/Dental/Vision | Significant equity upsideEntry-level Full TimeSan Francisco, CA, US16h ago
-
AI Intern USD 50K-58KAccess Control | Artificial Intelligence | Cybersecurity | Data Privacy | Help deskEntry-level InternshipYork, United States16h ago
-
Machine Learning Engineer USD 131K-178KAWS | Cassandra | Convolutional Neural Networks | Data Lakes | Data PipelinesMid-level Full TimeRemote, NY, US R16h ago
-
Amazon S3 | Data Engineering | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeNew York17h ago
-
Amazon S3 | Automation | Data Engineering | Data Modeling | Data Pipelines401k match | Dental insurance | Life insurance | Long-term disability | Medical insuranceSenior-level Full TimePrinceton17h ago
-
Senior Databricks Forward Deployed Engineer - GPS USD 119K-198KAPI Integration | AWS | Airflow | Azure | CI/CDTravelSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Atlanta, Georgia, …18h ago
-
Lead Databricks Forward Deployed Engineer - GPS USD 189K-372KAPI Integration | AWS | Airflow | Apache Spark | AzureSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Atlanta, Georgia, …18h ago
-
Lead AI and Data Solutions Engineer II USD 137K-229KAmazon Web Services | Apache Spark | Application Programming | Application Programming Interfaces | Cloud ComputingSenior-level Full TimeSacramento, California, United States; Tempe, Arizona, …18h ago
-
TikTok Shop - E-commerce Anti-Fraud Data Scientist USD 156K-296KA/B | A/B Testing | Analytics | B testing | Big DataMid-level Full TimeSeattle, Washington, United States18h ago