QA & UX Engineer

Who we are

Join our global Centre of Excellence as a subject matter expert (SME) for Agentic AI Services, collaborating with teams worldwide to drive innovation and excellence. As part of this central function, you will help shape, implement, and scale cutting-edge Agentic AI solutions across the organization and our clients

The AI Quality Assurance and Testing Engineer is an expert-level professional responsible for the strategy, planning, and execution of comprehensive testing for agentic AI systems including AI agents with complex decision-making capabilities.

This role ensures all AI solutions operate accurately, reliably, and ethically across both backend logic and end-user experiences (UI/UX). The role involves collaborating closely with project teams and business stakeholders to deliver robust, user-centric AI solutions and drive continuous improvement in QA practices for agentic AI. The individual also mentors and upskills less experienced team members in advanced AI and UX quality methodologies.

What you'll be doing

• Work with product owners, UX designers, data scientists, and developers to translate AI agent requirements and conversational/UI behaviors into comprehensive, testable scenarios and test plans.
• Design, develop, and execute test cases covering the full spectrum of agentic AI systems—including input/output logic, multi-turn dialog, reasoning, tool usage, action-taking, and integration with external APIs or RPA.
• Build and automate tests for both backend (models, workflow, knowledge bases) and frontend (UI/UX, conversational interfaces, visual feedback) components of AI agent solutions.
• Ensure thorough test coverage for user journeys—validating not only the technical correctness of the agents' outputs but also their usability, accessibility, responsiveness, and overall user experience.
• Develop and maintain automated testing pipelines using relevant QA frameworks and scripting (Python, Selenium, Playwright, Postman, or other pertinent tools) for both API-level and UI-level tests.
• Apply Responsible AI principles and implement test scenarios to identify issues related to fairness, transparency, safety, privacy, bias, and explainability in agentic AI behavior.
• Monitor online and post-release agent behavior to detect unexpected actions, degraded user experiences, or model drifts, and manage the reporting and remediation process.
• Collaborate with UX/UI teams to test and iterate on conversational design, error handling, and feedback mechanisms, ensuring seamless and intuitive interactions between users and AI agents.
• Analyze test results, track defects, provide recommendations for quality improvements, and report quality status across technical and business stakeholders, including executive leadership.
• Stay updated on the latest trends, frameworks, and tools in AI agent testing and UI/UX QA, proposing enhancements to tools and processes.
• Mentor and coach QA team members on advanced AI/ML and UI/UX testing best practices, fostering a culture of quality across agentic AI projects.
• Where needed, contribute to manual exploratory testing, especially for new agent flows or innovative interaction paradigms.

What you'll bring along

• Bachelor’s degree in Computer Science, Software Engineering, Human-Computer Interaction, Data Science, or related field.
• Relevant certifications in software testing (e.g., ISTQB), UX/UI (a plus), or AI/ML (a plus).
• Agile or DevOps certification advantageous.
• Minimum 5-7 years of experience in QA/testing roles on AI/ML products—ideally including chatbot/agentic AI, conversational AI, or virtual assistants.
• Strong background in UI/UX testing for modern web and/or mobile applications.
• Demonstrated experience with test automation frameworks for both backend (API/model) and frontend (UI) testing.
• Exposure to AI model lifecycle (training, inferencing, monitoring) and methods for post-deployment validation.
• Familiarity with tools and frameworks for accessibility and user experience validation.
• Experience planning and leading QA in Agile, multidisciplinary project teams, including mentoring junior QA engineers.
• Proven ability to contribute to the design and improvement of end-to-end testing processes for both AI systems and user-facing applications.
• Deep expertise in QA methodologies for AI/ML, particularly for conversational, task-driven, and agentic AI systems.
• Strong hands-on skills in both API and UI automation frameworks (such as Selenium, Playwright, or Cypress, and Python or JavaScript for scripting).
• In-depth knowledge of UI/UX best practices, heuristics, and accessibility standards, and how they intersect with AI-powered interfaces.
• Understanding of Responsible AI, including how to test for bias, explainability, and transparency in decisions and conversations.
• Excellent ability to design test strategies that bridge backend logic and user interaction layers for digital agents.
• Analytical and methodical, with outstanding troubleshooting, documentation, and communication abilities.
• Effective cross-functional collaborator and influential leader with a proactive, self-starting approach.
• Ability to distill complex findings for both technical and non-technical audiences.
• Excellent command of both spoken and written English.