AI Test and Evaluation Engineer

Looking for the opportunity to work on problems that matter, with colleagues that share your interest and expertise in AI and Machine Learning?  

Leidos is looking for a Test and Evaluation Engineer to apply their expertise to help Leidos research, develop, and commercialize test and evaluation techniques for AI systems. This includes leading the test and evaluation planning, process definition, tools and testing pipeline research, as well as verification and validation planning.

 We are looking for someone who is intellectually adaptive, likes to collaborate ad is inquisitive. You will work alongside junior and senior research/data scientists and data engineers with expertise in machine learning and artificial intelligence, who lead advanced research in AI.

You will work with a team of data scientists and data engineers to operationalize your research and make resulting tools and technologies deployable into operational environments.  You will contribute to new approaches for test and evaluation of AI models that produce measurable advances over state of the art solutions. 

Job Description

Looking for the opportunity to work on problems that matter, with colleagues that share your interest and expertise in AI and Machine Learning?  

Leidos is looking for a Test and Evaluation Engineer to apply their expertise to help Leidos research, develop, and commercialize test and evaluation techniques for AI systems. This includes leading the test and evaluation planning, process definition, tools and testing pipeline research, as well as verification and validation planning.

 We are looking for someone who is intellectually adaptive, likes to collaborate ad is inquisitive. You will work alongside junior and senior research/data scientists and data engineers with expertise in machine learning and artificial intelligence, who lead advanced research in AI.

You will work with a team of data scientists and data engineers to operationalize your research and make resulting tools and technologies deployable into operational environments.  You will contribute to new approaches for test and evaluation of AI models that produce measurable advances over state of the art solutions. 

Primary Responsibilities

• Develops Test and Evaluation CONOPS
• Creates scalable Test and Evaluation plans for AI systems including process, techniques and tools.
• Works with data scientists and developers to understand the AI system under test to develop test procedures
• Works with data scientists and developers to identify test data for positive and negative testing
• Works with MLOps engineers to integrate testing tools and procedures with the CI/CD pipeline
• Collects metrics to analyzes processes for potential improvements
• Collaborates with AI Governance team to maintain visibility and explainability through testing
• Implements testing process in the AI system design, development and deployment life cycle
• Identifies the risk in testing of projects, particularly for assessing the limitations of planned tests on complex AI systems
• Works within teams of AI/ML researchers and engineers using Agile development processes 

Basic Qualifications

• Bachelor's degree in Computer Science, Data Science or related field and 8+ years of relevant experience, Masters with 6+ years experience, or PhD with 4+ years experience.
• Strong Python programming fundamentals 
• Experience with system and subsystem level test process and automation
• Experience with creating user acceptance test scenarios
• Experience with SecDevOps tooling and MLOps pipeline development
• Experience with AI Performance and vulnerability assessment
• AI model assurance evaluation
• Experience applying and automating AI interpretability & explainability tools and methods
• Experience with developing CONOPS and presentations
• Good understanding of machine learning algorithms, tools and platforms
• Self-starter with high intellectual curiosity
• Great communication skills, able to explain model and test results to a non-technical audience
• Proficient in data exploration techniques and tools

Preferred Qualifications

• Experience with data visualization libraries such as Plotly, Streamlit, and matplotlib. 
• Experience with AI/ML tools, such as common Python packages (e.g., scikit-learn, NumPy, Pandas) and Jupyter notebooks 
• Experience with database administration and data repositories
• Experience in data exploration techniques and tools
• Experience with building LLM and other Generative AI applications. 
• Willing to learn new skills and platforms to support data analytics. 
• Ability and willingness to obtain a Top Secret security clearance 

Original Posting Date:

While subject to change based on business needs, Leidos reasonably anticipates that this job requisition will remain open for at least 3 days with an anticipated close date of no earlier than 3 days after the original posting date as listed above.

Pay Range:

The Leidos pay range for this job level is a general guideline only and not a guarantee of compensation or salary. Additional factors considered in extending an offer include (but are not limited to) responsibilities of the job, education, experience, knowledge, skills, and abilities, as well as internal equity, alignment with market data, applicable bargaining agreement (if any), or other law.