AppSierra Solutions - Senior Data Engineer - Python/PySpark

Overview

We are seeking a highly skilled Data Engineer with expertise in AWS, DataBricks, Snowflake, Python, and PySpark to join our dynamic team. In this role, you will be responsible for building and optimizing our data pipelines, ensuring high performance and reliability as we scale our data infrastructure. You will work closely with data scientists, analysts, and other stakeholders to create robust data solutions that support business intelligence and data analysis.

As a Data Engineer, you will harness the power of cloud technologies to streamline our data processes, and your proficiency in both Python and PySpark will be vital for developing and maintaining ETL workflows. Your contributions will empower the organization to make data-driven decisions while maintaining data integrity and security. You will have the opportunity to work with cutting-edge technologies within a collaborative and innovative environment, driving continuous improvement in our data management practices. We are looking for a proactive individual with a strong problem-solving mindset and a passion for working with large datasets to help us unlock the full potential of our data assets.

Responsibilities

• Design, build, and maintain scalable data pipelines using AWS, DataBricks, and Snowflake.
• Develop and implement ETL processes to extract, transform, and load data from diverse sources.
• Collaborate with data scientists and analysts to understand data needs and deliver appropriate data solutions.
• Optimize existing data workflows for performance and cost efficiency within cloud environments.
• Ensure data quality, integrity, and security through robust validation and monitoring processes.
• Document data architecture, data pipelines, and workflows to facilitate knowledge sharing and onboarding.
• Stay up-to-date with emerging technologies and best practices in data engineering and contribute to the technical strategy.
  
  

• Bachelor's degree in Computer Science, Engineering, or a related field.
• Minimum of 3 years of experience as a Data Engineer or in a similar role.
• Proficient in AWS services such as S3, Glue, Redshift, and EMR.
• Strong experience with DataBricks and Snowflake for data processing and storage.
• Expertise in Python and PySpark for scripting and data manipulation.
• Knowledge of data modeling techniques and best practices.
• Familiarity with data governance and compliance standards.
  
  

(ref:hirist.tech)