Lead Data Engineer - R01561332

Overview

• Azure Data Factory, Azure Data Lake, logic Apps and Azure SQL,MPP, PySpark, Databricks
  
  

Specialization

• Azure Data Engineering Advanced: Senior Data Engineer
  
  

Job requirements

• Job Description - Azure Data Engineer
• At least 4 to 6 years of relevant experience in data engineering team with 2-3 years hands-on experience in supporting Azure cloud technologies.
• Hands-on Expertise in Azure Data Engineering tool stack primarily Azure Data Factory, Azure Data Lake, logic Apps and Azure SQL,MPP (Azure Synapse) .
• Deep knowledge and hands-on on creating Azure Databricks notebooks (Good knowledge on PySpark, Databricks SQL, Delta Lake concepts) or Synapse Spark.
• Good understanding and exposure to Production Support of data integration and ETL systems with an understanding of SLAs, cross team collaboration and expectation setting.
• Good understanding and experience around Data Warehousing, ETL/ELT process, Relational Databases, Slowly changing dimension considerations etc.
• Should be familiar with Azure Synapse Workspace and good understanding of the Best practices for Data Loading to various target systems like Azure Synapse, data lake etc from various source systems.
• Understanding and able to build complex data transformation logic using Azure Databricks Pyspark for structured, semistructured data with different formats
• 8.Good understanding of No-SQL databases and experience with the same.
• Good understanding of SQL, Can write stored procedures, CTE queries, functions, understanding of normalization.
• Understanding of data loading from different source file formats (parquet, xml, json, csv etc), Experience with data Extraction from different data sources (Relational, FTP, SAP etc)
• Good understanding of security requirements of different Azure components
• Experience in best practices to resolve issues due to ingestion of varied data volumes by code related changes, optimal usage of clusters, partitioning and other methods
• Worked on 1-2 projects involving migration of large scale projects from On-Prem to cloud and also having a decent understanding of remediation of performance issues
• Understanding of Devops (CI\CD processes for automated deployments) and as applicable for data engineering (ADF, ADB, SQL deployments)
• Very good understanding of Agile to deliver consistent project outcome.
• Very high data affinity and strong analytical skills 17.Good understanding of data modelling concepts
• Analyzing and translating business needs into long-term solution data models
• Evaluating and analysis (reverse engineering) of existing data systems.
• Creation of Conceptual data models and data flows, Physical data models and data strategies
• Nice to have:- Certification in DP-203 (Microsoft Certified: Azure Data Engineer Associate), Databricks Data Engineering Certification