Data Engineer

Overview

Mandatory skills: Azure Data Factory, Azure Synapse, Azure Data bricks, Azure Fabric, Azure Data Lake Storage Gen 2 & Azure Blob storage, Delta Lake, CI/CD

Utilizes software engineering principles to deploy and maintain fully automated data transformation pipelines that combine a large variety of storage and computation technologies to handle a distribution of data types and volumes in support of data architecture design.

A Data Engineer designs data products and data pipelines that are resilient to change, modular, flexible, scalable, reusable, and cost effective.

· Design, develop, and maintain data pipelines and ETL processes using Microsoft Azure services (e.g., Azure Data Factory, Azure Synapse, Azure Data bricks, Azure Fabric).

· Utilize Azure data storage accounts for organizing and maintaining data pipeline outputs. (e.g., Azure Data Lake Storage Gen 2 & Azure Blob storage).

· Collaborate with data scientists, data analysts, data architects and other stakeholders to understand data requirements and deliver high-quality data solutions.

· Optimize data pipelines in the Azure environment for performance, scalability, and reliability.

· Ensure data quality and integrity through data validation techniques and frameworks. · Develop and maintain documentation for data processes, configurations, and best practices.

· Monitor and troubleshoot data pipeline issues to ensure timely resolution.

· Stay current with industry trends and emerging technologies to ensure our data solutions remain cutting-edge.

· Manage the CI/CD process for deploying and maintaining data solutions.

Job Type: Full-time

Application Question(s):

• How many years of experience in Data Engineer?
• how many years of experience in Azure Data Lake?
• how many years of experience in Azure Synapse?
• Are ok for Bangalore location?
• what is your current CTC?
• What is your expected CTC?
• What is your official notice period?

Work Location: In person