Sr. Data Engineer

The job responsibilities outlined in this document are not exhaustive and may evolve over time and be reviewed according to business needs.

ROLE DESCRIPTION SUMMARY

PRIMARY RESPONSIBILITIES / KEY RESULT AREAS

• Utilize a rigorous design methodology that consistently yields the right level of technical design documentation and develop technical solutions that conform both to customer requirements and information development standards.
• Building value-creating and innovative data solutions.
• Use a vast amount of programming tools to design new data solutions to ingest and integrate data into Unified Data Platform.
• Understand various requirement gathering and specification methodologies and can recommend one appropriate to a project.
• Designing, Implementing, and supporting a platform that can provide access to large datasets.
• Work as equals with Data Engineers, Quality Engineers, BI Developers and Business Analysts to establish optimal design/development/delivery which properly serves technical teams and our user community. Identify risks, tradeoffs, and investment decisions to facilitate best practices.

COMPETENCIES

• Strong problem-solving skills with acute attention to detail and ability to meet tight deadlines and project plans.
• Design and develop complex, large scale data structures and pipelines to organize, collect and standardize data that generate insights and address analytical/reporting needs.
• Design standards and best practices into new data engineering solutions.
• Proactively identify performance and data quality problems and drive to remediate them.
• Manage code versions and deployment process in source control and coordinate changes across team.
• Partner with Product Management and Operation teams to proactively identify monitoring gaps and improving upstream data collection and advocating for data driven thinking.
• Assist in running code reviews, guide team to follow standards, supporting team members in brainstorming and challenging designs.
• Solve and help other team members complex data issues around data integration of structured and unstructured sets other data processing incidents.

QUALIFICATIONS & EXPERIENCE

• Bachelor’s degree in Computer Science, Computer Engineering, Information Systems, or related technology.
• Minimum of 6 years of relevant data processing experience; 3+ years utilizing cloud-based services and data streaming.
• Experience with large scale data lake and data warehouse implementations in cloud environments.
• Expertise building data lakes and data warehouses in Snowflake using DBT, Phyton, Snowpark, Snowpipe and Snowflake SQL.
• Experience with Continuous Integration & Continuous Delivery (Jenkins)
• Experience in data analysis and writing and developing complex queries using SQL.
• Strong communication skills including technical documentation, development and delivery of demonstrations and presentations and the ability to listen and communicate effectively with functional leaders.

OTHER KEY REQUIREMENTS / COMMENTS

• Experience using variety of AWS cloud services like EC2, S3, Lambda.
• Hands on experience in DBT, Spark and Scala.
• Master’s in computer science, Engineering, or Information Systems
• Experience with Metadata management and Data Governance.
• Experience with Tableau, Power BI or similar data visualization tools
• Experience in airline / aviation, internet, or telecommunications