Software Engineer Lead-Python, Azure Databricks, Scala, Spark

Overview

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

About The Role

We are seeking a highly skilled and experienced Senior Software Engineer with expertise in Python, Databricks, Scala, Spark, and Azure to join our dynamic team. The ideal candidate will have a solid background in designing and developing large-scale data processing solutions and integrating them with cloud platforms. Experience with databases and SQL is required. Exposure to application development using .Net and React is a plus.

Primary Responsibilities

• Design, develop, optimize, and maintain scalable data pipelines using Databricks (Spark/Scala) on Azure
• Collaborate with cross-functional teams to deliver robust end-to-end data solutions
• Write efficient SQL queries for ETL processes, data validation, and reporting
• Optimize performance of complex data processing workflows
• Implement best practices for code quality, testing, deployment, and documentation
• Participate in code reviews and mentor junior engineers
• Work closely with stakeholders to understand business requirements and translate them into technical solutions
• Integrate data solutions with other applications
• Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
  
  

• Bachelor's or Master's degree in Computer Science or related field
• 6+ years of experience
• 6+ years of hands-on experience in software engineering or data engineering roles
• Solid experience with Databricks platform for big data analytics
• Hands-on experience working on Azure cloud services (e.g., Data Lake Storage, Azure Data Factory)
• Experience working within Agile or Scrum methodologies
• Solid background in Database design and development; excellent SQL skills (T-SQL or PLSQL etc.)
• Understanding of CI/CD pipelines for automated deployments on cloud platforms
• Proficient in Scala programming language; solid understanding of Spark framework (RDD or DataFrame or SQL)
  
  

• Exposure to .Net APIs or React-based UIs
• Exposure to application development using .Net (C# or ASP.NET) and/or ReactJS frontend frameworks
• Knowledge of RESTful API design or integration concepts
• Familiarity with DevOps tools like Azure DevOps or Jenkins or GitHub Actions
  
  

• Excellent analytical and problem-solving abilities
• Solid communication skills - able to convey technical concepts clearly to non-tech stakeholders
• Self-motivated team player who can work independently when required