Lead Data Engineer – AWS

Job Description

This role is ideal for a seasoned professional with experience in data engineering, team leadership, and project management. As a Lead AWS Data Engineer, you will be responsible for architecting scalable data solutions, leading engineering teams, and ensuring alignment with business objectives.

Roles and Responsibilities

• Provide technical leadership and mentorship to data engineering teams.
• Architect, design, and deploy scalable, secure, and high-performance data pipelines.
• Collaborate with stakeholders, clients, and cross-functional teams to deliver end-to-end data solutions.
• Drive technical strategy and implementation plans in alignment with business needs.
• Oversee project execution using tools like JIRA, ensuring timely delivery and adherence to best practices.
• Implement and maintain CI/CD pipelines and automation tools to streamline development workflows.
• Promote best practices in data engineering and AWS implementations across the team.

Required Skills

• Strong hands-on expertise in Python, PySpark, and Spark architecture, including performance tuning and optimization.
• Advanced proficiency in SQL and experience in writing optimized stored procedures.
  
• In-depth knowledge of the AWS data engineering stack, including:
  AWS Glue, Lambda, API Gateway, EMR, S3, Redshift, Athena
  
• Experience with Infrastructure as Code (IaC) using CloudFormation and Terraform.
  
• Familiarity with Unix/Linux scripting and system administration is a plus.
  
• Proven ability to design and deploy robust, production-grade data solutions.