Clinical AI Engineer

We are seeking a Clinical AI Engineer to join our innovative team working at the intersection of healthcare and AI. This role focuses on designing, evaluating, and improving LLM-powered tools that support abstraction and data analysis for clinical registries. The ideal candidate combines clinical domain expertise with a deep understanding of AI prompt engineering, model evaluation, and data quality analysis to build scalable solutions for real-world healthcare workflows.

As a Clinical AI Engineer, you’ll work closely with product, engineering, and clinical operations teams to define, test, and refine AI behaviors across a portfolio of registry automation tools. You’ll design prompts, evaluate AI outputs, and build systems that ensure safe, accurate, and performant abstraction across diverse patient populations and registry types.

Responsibilities:

Design and Evaluation of LLM Systems

• Develop and refine prompt strategies tailored to clinical registry questions and workflows.
• Design evaluation protocols and behavioral test suites to assess the performance of LLM-based systems for accuracy, relevance, and clinical appropriateness.
• Partner with ML/engineering teams to test novel prompting techniques, chain-of-thought reasoning, and retrieval augmentation in abstraction workflows.

Clinical and Registry Expertise

• Translate complex registry data requirements into structured abstraction workflows using LLMs.
• Apply clinical reasoning and judgement to identify subtle errors, gaps, and contextual misunderstandings in AI-generated outputs.
• Leverage experience with clinical data, abstraction processes, and EHR documentation to improve real-world usability and accuracy.

Data Quality, Analysis, and Measurement

• Design metrics and dashboards to measure model coverage, abstraction precision/recall, and user correction rates.
• Build error taxonomies and track failure modes across registry products to drive continuous improvement.
• Collaborate with engineering and research teams to recommend architectural or training enhancements to LLM systems.
  
  

Product and Team Collaboration

• Contribute clinical insight and feedback to roadmap development for registry abstraction and data quality tooling.
• Work closely with nurses, abstractors, and clinical informaticists to validate abstraction behavior and usability.
• Pair with software engineers and AI researchers to test product hypotheses and bring features to production.

What you’ll need:

Clinical Knowledge & Experience

• Strong foundation in interpreting patient data including diagnoses, labs, procedures, and treatment timelines.
• Experience working directly with registry programs (e.g., STS, NSQIP, ACC, etc.) and abstracting clinical data.
• Familiarity with nursing workflows and how clinicians interact with EHRs and abstraction interfaces.

AI Prompt Engineering and Evaluation

• 1+ year hands-on experience designing prompts for LLMs in a professional or research setting.
• Strong intuition for how AI systems interpret instructions and when/how they fail in clinical contexts.

Experience designing test sets and evaluation pipelines for AI output behavior.

Data QA and Measurement

• Experience measuring performance of AI or human-in-the-loop systems with precision, recall, and coverage metrics.
• Familiarity with clinical data QA principles and constructing reproducible test environments.

Ability to create and use error taxonomies to drive iterative improvement of AI behaviors.

Additional Skills

• Excellent written and verbal communication with both technical and clinical stakeholders.
• Comfortable working in fast-paced, cross-functional teams where responsibilities may span research, engineering, and operations.

Bachelor’s degree in a clinical field (e.g., Nursing, Health Informatics, Public Health) or equivalent experience. Master’s degree or relevant AI coursework is a plus.

Why we love Carta Healthcare, and why you will too!

• Working on a new, challenging problem space with LLMs/AI/Healthcare
• Industry leading products
• Work hard, and have fun doing it
• Work alongside some of the most talented and dedicated teammates
• Mission driven 
• Competitive benefits package including great healthcare benefits and 401k

All applicants require legal authorization to work in the United States now or in the future without sponsorship.

Salary Range: $89,900 to $97,900

In compliance with federal law, all persons hired will be required to verify identity and eligibility to work in the United States and to complete the required employment eligibility verification form upon hire. All applicants are required to reside within the continental United States. 

Carta Healthcare is dedicated to building a diverse and inclusive company because we serve health systems across the country; we’ve seen how our product and impact are strengthened the more we reflect that diversity. In addition, we have found and strongly believe that diverse teams are higher-performing, and we embrace the varied perspectives that our team members share with each other. As such, we are an Equal Opportunity Employer. 

#LI-Remote #BI-Remote