Principal Software Engineer

Overview

Make a difference in the technology that is at the core of some of the most innovative and influential businesses in the world. Red Hat’s Multi-product and Engineering Experience (MPEX) Integrity Engineering organization is looking for a highly skilled Principal Software Engineer focused on Chaos Engineering to join our team in India.

In this role, you will work at the forefront of reliability, resilience, and failure testing for Red Hat’s Hybrid Platforms. You will serve as a technical architect and leader for Chaos Engineering efforts, defining Chaos Engineering standards and practices for Hybrid Platforms and influencing platform reliability decisions across multiple product areas.

You will help create robust, production-ready systems by validating real-world failure scenarios and ensuring platform stability across complex, multi-product environments. This role requires strong software development and testing skills, deep Linux or Unix expertise, and experience designing and operating large, distributed, and layered systems. Collaboration is central to Red Hat’s open-source culture, and you will work closely with engineering, product, QE, and SRE teams to scale Chaos Engineering adoption and drive measurable improvements in customer-facing reliability.

What Will You Do

• Define, document, and communicate a clear vision for Chaos Engineering aligned with strategic initiatives across the Hybrid Platforms organization.
• Design and implement Chaos testing strategies focused on real-world failure modes, customer scenarios, and multi-product interactions.
• Technically lead Hybrid Platforms teams in identifying, reproducing, and resolving resilience and stability issues uncovered through Chaos testing.
• Collaborate closely with engineering, product, QE, and SRE stakeholders to close gaps in resilience coverage.
• Integrate Chaos testing into CI pipelines such as OpenShift CI to enable early detection of regressions and reliability risks.
• Execute upgrade and lifecycle Chaos testing to validate platform behavior during disruptive operations and transitions.
• Document defects uncovered through Chaos testing and actively advocate for their prioritization and resolution.
• Create and maintain dashboards and reports to track Chaos test coverage, outcomes, and reliability trends.
• Clearly communicate customer and business impact of resilience findings to technical and non-technical stakeholders.
• Stay current with industry trends, tools, and best practices in Chaos Engineering and distributed systems.
• Enhance testing infrastructure, automation, and observability to scale Chaos testing adoption across teams.
  
  

• 10+ years of experience in software engineering, SRE, platform engineering, quality engineering, or reliability-focused roles.
• Strong Linux or Unix system administration and networking skills, including experience with Git-based workflows.
• Hands-on experience with containers and orchestration platforms such as Red Hat OpenShift and Kubernetes.
• Experience working with large-scale distributed systems and virtualization technologies such as KVM, VMware, or Libvirt.
• Advanced programming skills in Python, including building and maintaining automated testing frameworks and tooling.
• Experience implementing and integrating testing workflows into CI systems, preferably OpenShift CI.
• Proven ability to technically lead and mentor engineers through complex, cross-team initiatives.
• Strong written and verbal communication skills in English, with experience collaborating across globally distributed teams.
  
  

• Experience with Chaos Engineering tools and frameworks such as Krkn or similar platforms.
• Familiarity with automation and configuration management tools such as Ansible.
• Knowledge of open-source development models and communities.
• Experience working in agile development environments.
• Certifications such as ISTQB, CSTE, or related credentials are optional.
  
  

# LI-CS1

About Red Hat

Red Hat is the world’s leading provider of enterprise open source software solutions, using a community-powered approach to deliver high-performing Linux, cloud, container, and Kubernetes technologies. Spread across 40+ countries, our associates work flexibly across work environments, from in-office, to office-flex, to fully remote, depending on the requirements of their role. Red Hatters are encouraged to bring their best ideas, no matter their title or tenure. We're a leader in open source because of our open and inclusive environment. We hire creative, passionate people ready to contribute their ideas, help solve complex problems, and make an impact.

Inclusion at Red Hat

Red Hat’s culture is built on the open source principles of transparency, collaboration, and inclusion, where the best ideas can come from anywhere and anyone. When this is realized, it empowers people from different backgrounds, perspectives, and experiences to come together to share ideas, challenge the status quo, and drive innovation. Our aspiration is that everyone experiences this culture with equal opportunity and access, and that all voices are not only heard but also celebrated. We hope you will join our celebration, and we welcome and encourage applicants from all the beautiful dimensions that compose our global village.

Equal Opportunity Policy (EEO)

Red Hat is proud to be an equal opportunity workplace and an affirmative action employer. We review applications for employment without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, citizenship, age, veteran status, genetic information, physical or mental disability, medical condition, marital status, or any other basis prohibited by law.

Red Hat does not seek or accept unsolicited resumes or CVs from recruitment agencies. We are not responsible for, and will not pay, any fees, commissions, or any other payment related to unsolicited resumes or CVs except as required in a written contract between Red Hat and the recruitment agency or party requesting payment of a fee.

Red Hat supports individuals with disabilities and provides reasonable accommodations to job applicants. If you need assistance completing our online job application, email application-assistance@redhat.com. General inquiries, such as those regarding the status of a job application, will not receive a reply.