Azure Data Engineer

Overview

Azure Data Engineer -

3 to 8 years of experience in developing data engineering and analytics pipelines with a minimum of 1 years of core experience in MS Azure and Databricks. Strong development and problem-solving skills. Experienced in development of Data platforms / Datalakes using ETL or ELT approach, preferably Databricks Delta lakehouse. Development of real-time data workflows and pipelines using (Py)Spark Structured Steaming solutions. Must have programming skills: Python, PySpark, SQL Must have PostgreSQL database. A fair understanding of commonly used data offerings on Azure (ADF, Synapse etc). Knowledge of DevOps processes and hands-on experience in implementing pipelines using CI/CD methodology. They should be able to explain the data pipeline architecture and flow they have worked on, including the types of transformations they are performing and how they are implementing them.Explain about the types of actions and transformations in Spark.Spark architectureWhat are the optimization techniques in data bricks and spark.Coding questions likeconvert Json to dataframe using spark and vice versa.About wide and narrow transformations.Convert nested json to dataframeAbout transformations and actions like map, filter etc.,Medallion architectureAbout delta tables.How do they handle the incremental load in pipelines.SQL questions like how they remove duplicates and other basic but they should be above write the queries properly.

Job Types: Full-time, Permanent

Pay: Up to ₹1,938,901.65 per year

Schedule:

• Day shift

Work Location: In person

Application Deadline: 28/03/2025