Overview
Job Title: Senior QA Engineer – Web, Mobile & Conversational AI Testing
Location: Remote
Experience Level: 4+ years
Employment Type: Contract
About the Role:
We are looking for a highly experienced Senior QA Engineer to lead and execute quality assurance efforts across our web, mobile, and AI-driven conversational products. You will play a critical role in ensuring our applications deliver exceptional user experiences, high performance, and reliability—especially in LLM-based (Large Language Model) conversational systems.
Key Responsibilities:
-
Test Planning & Strategy
-
Develop and maintain comprehensive test plans and test cases for web, mobile, and conversational AI applications.
-
Define QA strategy for end-to-end testing of features including functional, UI/UX, performance, and regression testing.
-
Execution & Automation
-
Perform manual and automated testing for mobile (iOS/Android) and web platforms. [Selenium, Pytest, Python Scripts]
-
Design and maintain automated test scripts using tools like Selenium, Appium, or Playwright.
-
Conduct conversational flow and response accuracy testing for LLM-based chat interfaces.
-
LLM & Conversational App Testing
-
Evaluate dialogue quality, prompt-response relevance, tone, and fallback behaviors in GPT-like systems.
-
Test for hallucinations, safety, toxicity, and coherence in AI outputs.
-
Validate integrations with RAG systems, vector stores, and API workflows.
-
Cross-Functional Collaboration
-
Collaborate with product managers, developers, and AI engineers to ensure feature clarity and testability.
-
Provide feedback early in the development lifecycle to reduce bugs and enhance quality.
-
Monitoring & Reporting
-
Track and report bugs using tools like JIRA or Linear.
-
Analyze and report on test results, coverage, and quality trends.
Required Qualifications:
-
4+ years of experience in QA for web and mobile applications.
-
Hands-on experience with both manual and automation testing tools and frameworks.
-
Strong knowledge of QA methodologies, testing types, and software development lifecycle.
-
Automation testing using Pytests.
-
Experience testing LLM-based or NLP-powered applications (ChatGPT, Rasa or custom LLMs).
-
Proficient in writing SQL queries and working with Postman for API testing.
-
Familiarity with vector databases and conversational testing tools (Langfuse, Phoenix, etc.) is a plus.
Preferred Skills:
-
Experience in testing AI-driven personalization, intent recognition, or multi-turn conversation logic.
-
Exposure to prompt engineering and evaluation frameworks for LLM outputs.
-
Familiarity with CI/CD pipelines and version control systems like Git.
Ability to build QA dashboards and automated quality reports.