Pune, Maharashtra, India
Information Technology
Full-Time
Relanto
Overview
Job Title: QA Engineer – Generative AI Location: India Experience Level: 3–6 years
Job Summary
We are looking for a QA Engineer with a passion for ensuring the quality and reliability of AI-powered applications, especially those built using Generative AI technologies. You will play a critical role in validating AI systems, including LLMs, RAG pipelines, and backend services. Your work will help shape the safety, accuracy, and robustness of AI-driven features used by thousands (or millions) of users.
Key Responsibilities
Functional Testing
Job Summary
We are looking for a QA Engineer with a passion for ensuring the quality and reliability of AI-powered applications, especially those built using Generative AI technologies. You will play a critical role in validating AI systems, including LLMs, RAG pipelines, and backend services. Your work will help shape the safety, accuracy, and robustness of AI-driven features used by thousands (or millions) of users.
Key Responsibilities
Functional Testing
- Design, execute, and maintain test cases for APIs, backend services, and UI interfaces.
- Conduct regression, integration, and exploratory testing of AI-powered features.
- Validate RESTful APIs and end-to-end workflows in web applications.
- Test outputs from LLMs and GenAI pipelines for factual accuracy, coherence, tone, and safety.
- Evaluate outputs for hallucination, repetition, bias, toxicity, and other model behavior anomalies.
- Create structured test prompts and scenarios to assess the consistency and reliability of AI responses.
- Perform adversarial and edge case testing, including prompt injection and jailbreak scenarios.
- Develop and maintain automation test suites using tools like Selenium, Playwright, Cypress, or PyTest.
- Automate API testing workflows using Postman, REST-assured, or similar tools.
- Integrate test automation into CI/CD pipelines for continuous validation and fast feedback loops.
- Conduct performance, load, and stress testing on model inference APIs and backend services.
- Measure and validate latency, throughput, and resource utilization for GenAI features.
- Test system behavior under scale and ensure failover and resilience capabilities.
- Collaborate with data scientists to verify training data integrity and model evaluation results.
- Validate semantic search results in vector databases used in RAG (Retrieval-Augmented Generation) pipelines.
- Test chaining logic and prompt orchestration using frameworks like LangChain.
- Perform tests to identify data privacy risks, prompt injections, and leakage of sensitive content.
- Support red-teaming efforts to proactively identify vulnerabilities in GenAI systems.
- Contribute to documentation of risk mitigation plans and compliance readiness for audits.
- Bachelor’s degree in Computer Science, Engineering, or related field.
- 3+ years of professional experience in software QA; exposure to AI/ML projects is a plus.
- Strong understanding of QA methodologies, test planning, and execution.
- Hands-on experience testing APIs, backend systems, and web applications.
- Proficiency with test automation frameworks like Selenium, Playwright, Cypress, or PyTest.
- Experience with API testing tools like Postman, REST-assured, or similar.
- Familiarity with LLMs, NLP concepts, and GenAI tools (e.g., OpenAI, Hugging Face, LangChain).
- Working knowledge of Git, CI/CD tools, and test management platforms.
Similar Jobs
View All
1 Day ago
TrueFan - Senior Machine Learning Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
About UsTrueFan is at the forefront of AI-driven content generation, leveraging cutting-edge generative models to build next-generation products. Our mission is to redefine content generation space through advanced AI technologies, including deep ge...
1 Day ago
Salesforce commerce cloud consultant
Information Technology
- Thiruvananthapuram, Kerala, India
Salesforce Commerce Cloud consultant 5+ Years of Experience 6 to 12 months Mode - Remote 1.1LPM - 1.2LPM Max Key Responsibilities Translate business requirements into scalable Salesforce Service Cloud solutions, in collaboration with CAE's technic...
1 Day ago
Cloud Infrastructure Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
DescriptionInvent the future with us. Recognized by Fast Company’s 2023 100 Best Workplaces for Innovators List, Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focuse...
1 Day ago
Devops Engineer- Intermetiate
Information Technology
- Thiruvananthapuram, Kerala, India
BackJD: Dev ops Engineer:As a DevOps Specialist- should be able to take ownership of the entire DevOps process, including Automated CI/CD pipelines and deployment to production.They should also be comfortable with risk analysis and prioritization.Le...
1 Day ago
Sr Data Scientist (London)
Information Technology
- Thiruvananthapuram, Kerala, India
AryaXAI stands at the forefront of AI innovation, revolutionizing AI for mission-critical, highly regulated industries by building explainable, safe, and aligned systems that scale responsibly. Our mission is to create AI tools that empower research...
1 Day ago
Software Test Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further att...
1 Day ago
Software Developer 5 (Java Fullstack)
Information Technology
- Thiruvananthapuram, Kerala, India
Job DescriptionBuilding off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team focuses on product development and product strategy for Oracle Health, while building out a complete platfo...
1 Day ago
Java Developer - Spring Frameworks
Information Technology
- Thiruvananthapuram, Kerala, India
Java DescriptionWe are looking for a passionate and talented Java Developer with 2-3 years of hands-on experience to join our growing development team.The ideal candidate should have a strong foundation in Java technologies and the ability to develo...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in