Senior QA Engineer – Web, Mobile & Conversational AI Testing

Job Title: Senior QA Engineer – Web, Mobile & Conversational AI Testing 
Location: Remote 
Experience Level: 4+ years 
Employment Type: Contract 

About the Role: 

We are looking for a highly experienced Senior QA Engineer to lead and execute quality assurance efforts across our web, mobile, and AI-driven conversational products. You will play a critical role in ensuring our applications deliver exceptional user experiences, high performance, and reliability—especially in LLM-based (Large Language Model) conversational systems. 

Key Responsibilities: 

• Test Planning & Strategy 
  

• Develop and maintain comprehensive test plans and test cases for web, mobile, and conversational AI applications. 
  

• Define QA strategy for end-to-end testing of features including functional, UI/UX, performance, and regression testing. 
  

• Execution & Automation 
  

• Perform manual and automated testing for mobile (iOS/Android) and web platforms. [Selenium, Pytest, Python Scripts] 
  

• Design and maintain automated test scripts using tools like Selenium, Appium, or Playwright. 
  

• Conduct conversational flow and response accuracy testing for LLM-based chat interfaces. 
  

• LLM & Conversational App Testing 
  

• Evaluate dialogue quality, prompt-response relevance, tone, and fallback behaviors in GPT-like systems. 
  

• Test for hallucinations, safety, toxicity, and coherence in AI outputs. 
  

• Validate integrations with RAG systems, vector stores, and API workflows. 
  

• Cross-Functional Collaboration 
  

• Collaborate with product managers, developers, and AI engineers to ensure feature clarity and testability. 
  

• Provide feedback early in the development lifecycle to reduce bugs and enhance quality. 
  

• Monitoring & Reporting 
  

• Track and report bugs using tools like JIRA or Linear. 
  

• Analyze and report on test results, coverage, and quality trends. 
  

Required Qualifications: 

• 4+ years of experience in QA for web and mobile applications. 
  

• Hands-on experience with both manual and automation testing tools and frameworks. 
  

• Strong knowledge of QA methodologies, testing types, and software development lifecycle. 
  

• Automation testing using Pytests. 
  

• Experience testing LLM-based or NLP-powered applications (ChatGPT, Rasa or custom LLMs). 
  

• Proficient in writing SQL queries and working with Postman for API testing. 
  

• Familiarity with vector databases and conversational testing tools (Langfuse, Phoenix, etc.) is a plus. 
  

Preferred Skills: 

• Experience in testing AI-driven personalization, intent recognition, or multi-turn conversation logic. 
  

• Exposure to prompt engineering and evaluation frameworks for LLM outputs. 
  

• Familiarity with CI/CD pipelines and version control systems like Git. 
  

• Ability to build QA dashboards and automated quality reports.