Data Engineer

Overview

As an Data Engineer at Gainwell, you can contribute your skills as we harness the power of technology to help our clients improve the health and well-being of the members they serve — a community’s most vulnerable. Connect your passion with purpose, teaming with people who thrive on finding innovative solutions to some of healthcare’s biggest challenges. Here are the details on this position. This role is strictly involved in the development of the product and does not involve access to Protected Health Information (PHI) & Personally Identifiable Information (PII) or any secured/confidential client data. The work is limited to application development and does not include handling or processing of sensitive health information.

Your role in our mission

• Lead the design and development of highly scalable, robust data pipelines and ETL/ELT solutions using Databricks and Apache Spark.
• Provide expertise in Databricks workspace, cluster, and job management, focusing on performance tuning, cost optimization (DBU reduction), autoscaling, and scheduling.
• Define and enforce technical standards, design patterns, and best practices for data modelling, quality, and schema management across the platform.
• Drive the adoption of DataOps principles by integrating Databricks workloads with enterprise CI/CD tools (e.g., Azure DevOps, GitHub Actions) for automated code and model lifecycle management.
• Act as the technical liaison, engaging with stakeholders to gather requirements, communicating complex designs, and ensuring data solutions meet business needs for quality and consistency.
• Provide technical guidance and mentorship to junior team members, lead project planning and estimation, and create clear architecture and process documentation.
• Focus on maintaining, debugging, and optimizing existing data systems to ensure reliability and low technical debt.
• Ensure data extraction, transformation, loading, normalization, and cleansing.
• Collaborate on the development and documentation of data models, schemas, and source-to-target mappings within the data warehouse.
• Conceptualize and visualize scalable data frameworks.
• Communicate effectively with internal and external stakeholders to support project objectives.
• Demonstrated expertise in the Databricks platform (AWS/Azure), including deep knowledge of Spark fundamentals, Delta Lake architecture, and Unity Catalog.
• Strong knowledge of cloud platforms (AWS preferred) and advanced proficiency in SQL, data modelling, data warehousing concepts, and SQL optimization.
• Proven experience with CI/CD tools and hands-on implementation of DataOps and MLOps principles.
• Familiarity and experience in implementing data governance and data quality practices.
• Demonstrated ability in project planning, low-level design (LLD), accurate estimation, and managing project delivery at a team level.
  
  

• 9+ years of overall experience with minimum 5 years’ experience with big data technologies on AWS, Azure.
• Bachelor’s degree in computer science or a related field, or equivalent education.
• At least 3 years’ hands-on experience with Databricks or Apache Spark (Python/Scala).
• Proficient in working with various database structures, including transactional and data warehouse environments.
• Databricks and AWS developer/architect certifications are highly desirable.
• Demonstrated ability to lead and mentor engineering teams.
• Excellent communication abilities.
• Capacity to deliver impactful results across multiple concurrent projects.
• Strategic thinker with the ability to balance immediate and long-term objectives.
  
  

• Work environment: Remote