Senior Software Engineer I

Overview

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

Primary Responsibilities

• Design and execute test plans for LLM and agentic features to ensure accuracy and safety
• Perform robustness testing, including adversarial, prompt injection, and RAG validation
• Implement bias, fairness, and toxicity checks using observability tools like Arize and Phoenix
• Monitor model drift and performance trends, providing compliance evidence
• Prepare technical and regulatory documentation for governance and review boards
• Act as a quality gatekeeper, ensuring AI solutions meet ethical and patient safety standards before production
• Mentor junior team members on AI testing best practices and compliance frameworks
• Define and track quality metrics (accuracy, robustness, compliance) and produce regular reports for leadership
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
  
  

• Undergraduate degree or equivalent experience
• Experience in software quality engineering, with at least 1+ years in AI/ML testing
• Experience with robustness testing (adversarial, prompt injection) and RAG validation
• Experience preparing technical documentation for compliance and governance
• Solid understanding of AI/ML validation, including LLM and agentic feature testing
• Familiarity with bias, fairness, and toxicity checks using observability tools like Arize and Phoenix
• Knowledge of regulatory and ethical standards for AI in healthcare
• Proficiency in test automation frameworks, scripting (Python), and CI/CD integration
• Proven ability to mentor junior engineers and promote best practices
• Proven ability to monitor model drift, analyze performance trends, and support compliance reporting
• Proven skills in tracking quality metrics and generating reports for leadership
  
  

• AI Dojo Certification or willingness to complete advanced AI training
• Exposure to LLMs, agentic architectures, and AI observability tools
  
  

#njp #nic