Artificial Intelligence Assurance Researcher

What does AI/ML assurance research look like in this position?

We collaborate with civilian government agencies to test and evaluate military AI/ML-enabled technology to provide our warfighters with safe, effective, and usable technology. We work on a variety of efforts, including:

• Authoring test and evaluation guidance and educational resources for AI/ML that include performance, security, integration, ethical, and DoD acquisition considerations;
• Monitoring AI/ML advancements across the DoD by interviewing program offices and attending conferences and workshops to inform investments in testing infrastructure;
• Translating and adapting AI/ML testing methods, including those from industry and academia, to address challenges posed by complex use cases and environments alongside various time, monetary, and safety constraints;
• Providing AI/ML guidance and consultations to government sponsors and IDA researchers to inform AI/ML evaluations across diverse technologies and use cases;
• Communicating project nuances and analytical results via writing and presentations to diverse audiences, including government and military leadership, system end-users, and technical audiences without AI/ML expertise; and
• Identifying AI/ML testing challenges posed by DoD use cases and contributing to strategic-level efforts to advocate for AI/ML assurance best practices throughout the DoD.

What qualifications and skillsets do you need?

Required:

• PhD or MS with 6+ years of relevant experience. We value diverse expertise and will consider applicants from a wide array of disciplines, including but not limited to:
  • Responsible AI (RAI), AI Ethics, Data Management, Science and Technology Policy, Artificial Intelligence, Machine Learning, Data Science, Computer Science, Operations Research, and STEM backgrounds with AI/ML application experience.
• Foundational understanding of common machine learning algorithms, including their limitations and failure modes.
• Proficiency with at least one common scripting language (i.e., Python, R);
• Availability to travel (approx. 10% time); and
• Ability to obtain or currently holds a security clearance;

Preferred:

• Experience consulting/advising groups in AI/ML development, testing, and/or governance; 
• Experience with AI/ML in complex, high-risk use cases with messy data and/or poorly defined objectives; and 
• Experience developing and evaluating generative AI and/or computer vision models.

Why work at IDA?

IDA, as a non-profit, is uniquely positioned to answer the most challenging U.S. security and science questions with objective and rigorous analysis leveraging extraordinary scientific, technical, and analytic expertise. This position is located in Alexandria, VA, just outside Washington, D.C.

IDA offers the following benefits and resources to their employees:

• Comprehensive benefits including diverse health insurance options, generous 11% contribution to retirement, maternity/paternity leave, 20 days paid time off, and relocation reimbursement;
• Mission-oriented work at a non-profit that supports and protects our warfighters; as a non-profit, IDA employees are eligible for Public Service Loan Forgiveness (PSFL);
• Professional development including tuition reimbursement, internal and external trainings, and funding for researcher-pitched projects and publications;
• A culture of work-life balance, including flex time, partial telework, and limiting work communication to core business hours;
• A brand-new facility located in Alexandria’s high-tech corridor in Potomac Yard; amenities include a cafeteria, gym, and close proximity to local dining and shopping; and
• Easy access to Potomac Yard Metro stop and Potomac Yard DASH bus stop. Employer contributes $315/month (IRS max) to commuter benefits.

What are the next steps?

• Complete the application and submit your resume.

The application process consists of three interviews.

1. A short screening call with HR;
2. A 30 min – 60 min screening call with researchers; and
3. A full day interview with various researchers and staff; includes giving an hour presentation on a research project.

Prior to the full day interview, you will be asked to provide:

• Transcripts (unofficial accepted);
• 3 letters of recommendation; and
• A writing sample.