AI QA Engineer

It's fun to work at a company where people truly believe in what they are doing!

Job Description:

Position Summary

The QA/Support Engineer works as a member of the Operations team within Epiq’s Legal Solutions division. In this role, the QA/Support Engineer plays a crucial role at Epiq by bridging quality assurance and customer/technical support, ensuring our software is tested thoroughly, reliable in production and backed by excellent technical assistance for both internal and external users.

You will work closely with the product owners, developers and other support staff to design test cases, identify and document bugs, reproduce issues and help deliver high-quality solutions. This role is ideal for someone who enjoys problem solving, has strong attention to detail and thrives in a fast-paced collaborative environment.

Essential Job Responsibilities

• Test Design & Automation
• Design, write, and execute manual and automated test cases for both traditional and AI/ML features.
• Use Power Automate to streamline repetitive QA workflows (log collection, test execution triggers, defect triage).
• Leverage GitHub Copilot to accelerate writing automated test scripts, SQL queries, and debugging code.
• AI-Specific Validation
• Validate AI outputs for accuracy, explainability, and fairness using AI testing frameworks.
• Document test results and issues clearly, using AI-assisted summarization (e.g., Copilot) to improve speed and clarity.
• Complex Data & Relationship Testing
• Use Lean Graph to model and validate relationships within large, interconnected legal datasets.
• Test data-driven scenarios such as permissions, document linkages, and case dependencies to ensure accuracy and consistency.
• Simulation & Real-World Workflow Testing
• Employ Sim Studio (or similar simulation tools) to replicate end-to-end user journeys and stress test at scale.
• Simulate edge cases (network failures, AI misclassifications, unusual input data) to validate system robustness.
• Monitoring & Support
• Utilize monitoring, logging, and alerting systems to proactively identify anomalies in AI and non-AI modules.
• Collaborate with internal/external clients, engineers, and developers to isolate, debug, and resolve AI-related issues.
• Create documentation, troubleshooting guides, and FAQs — with AI assistance (Copilot, Power Automate flows) to speed delivery.
• Continuous Improvement
• Contribute to the evolution of QA processes by embedding AI-driven test automation and simulation practices.
• Ensure QA best practices extend to AI explainability, data quality validation, and fairness testing.
  

Requirements

• Bachelor’s degree in computer science, engineering, or related field (or equivalent experience).
• 2+ years of experience in Quality Assurance and Automation Testing, including at least 1 year of hands-on exposure to QA methodologies (unit, integration, regression, UAT) with a focus on their application in AI/ML testing environments.
• Strong experience with bug tracking, project management, and logging/debugging tools.
• Proficiency with databases (MS SQL, Snowflake, Oracle, MySQL) for test data design and validation.
• Ability to work in fast-paced environments, managing multiple priorities with high attention to detail.
• Excellent communication skills, with the ability to explain technical/AI concepts to non-technical audiences.
  
  

Tools & Technologies

Mandatory

• Automation & Productivity: Power Automate, GitHub Copilot, Selenium, Robot Framework
• Databases: MS SQL, Snowflake, Oracle, MySQL
• Monitoring & Logging: Splunk, ELK Stack (Elastic, Logstash, Kibana)
• Programming/Scripting: C#, Python, .NET basics
• Cloud Environments: Azure or AWS (for test environments and AI integrations)
  
  

Good-to-Have

• AI/ML Testing & Validation: Great Expectations, Evidently AI, Fairlearn, AIF360, MLflow
• Data & Graph Testing: Lean Graph (Neo4j), Pandera, Deequ
• Simulation & Workflow Testing: Sim Studio, JMeter, LoadRunner
• Explainability & Debugging: SHAP, LIME, Captum, Alibi
• Advanced Monitoring/AI Ops: WhyLabs, Arize AI
  
  

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

It is Epiq’s policy to comply with all applicable equal employment opportunity laws by making all employment decisions without unlawful regard or consideration of any individual’s race, religion, ethnicity, color, sex, sexual orientation, gender identity or expressions, transgender status, sexual and other reproductive health decisions, marital status, age, national origin, genetic information, ancestry, citizenship, physical or mental disability, veteran or family status or any other basis protected by applicable national, federal, state, provincial or local law. Epiq’s policy prohibits unlawful discrimination based on any of these impermissible bases, as well as any bases or grounds protected by applicable law in each jurisdiction. In addition Epiq will take affirmative action for minorities, women, covered veterans and individuals with disabilities. If you need assistance or an accommodation during the application process because of a disability, it is available upon request. Epiq is pleased to provide such assistance and no applicant will be penalized as a result of such a request. Pursuant to relevant law, where applicable, Epiq will consider for employment qualified applicants with arrest and conviction records.