Senior Lead Data Engineer

Overview

Key Skills: Data Warehouse, SQL, Pyspark, AWS, Python, Java

Roles and Responsibilities:

• Leading projects: Working on complex, cross-functional projects, and acting as a subject matter expert
• Building data Solution: Creating secure, stable, and scalable data pipelines, managing analytics platforms
• Design Standards and Framework: Provides appropriate consulting, interfacing, standards and framework for data pipelines,
• Advising Junior Data Engineers: Providing guidance to the fellow data engineers, operations support and implementation partners and making recommendations
• Evaluating access control: Evaluating and reporting on access control processes to determine data asset security
• Developing documentation: Creating technical documentation that supports best practices

Skills Required:

• Experience in preparing Data warehouse design artifacts based on given requirements - Defining Standards, Building Frameworks, and Source-Target Mapping
• Strong analytical and data modeling skills including logical and physical schema design
• Deep understanding of database technology and data use within analytic platforms
• Strong programming skills with SQL and PySpark,
• Experience with AWS cloud Services including EMR, Redshift/Postgres
• Experience in other Cloud / ETL technologies including Databricks, Snowflakes, Ab Initio
• Good Understanding of other programming languages Python, Java
• Proven experience with large, complex database projects in environments producing high-volume data
• Demonstrated Problem solving skills; familiarity with various root cause analysis methods; experience in documenting identified problems and determined resolutions.
• Experience in analyzing Performance Issues analysis and Tuning

Preferred Qualifications:

• Proven experience (15 years minimum) with Datawarehouse design, build performance tuning and optimization
• Very good knowledge of data warehouse architecture approaches and trends, and high interest to apply and further develop that knowledge, including understanding of Dimensional Modelling and ERD design approaches,
• Experience in developing streaming applications Spark Streaming, Flink, Storm, etc.
• Excellent conceptual abilities compared with very good technical documentation skills, e.g. ability to understand and document complex data flows as part of business / production processes,
• Familiarity with SDLC concepts and processes

Additional Skill:

• Experience in semiconductor / High Tech Manufacturing industry
• Experience with reporting data solutions and business intelligence tools
• Knowledge of statistical data analysis and data mining
• Experience in test management, test case definition and test processes

Education: Master's degree in information technology, Electrical Engineering or similar relevant fields.