Sr. Data Engineer

Overview

• Design and build data pipelines using Spark-SQL and PySpark in Azure Databricks
• Design and build ETL pipelines using ADF
• Build and maintain a Lakehouse architecture in ADLS / Databricks.
• Perform data preparation tasks including data cleaning, normalization, deduplication, type conversion etc.
• Work with DevOps team to deploy solutions in production environments.
• Control data processes and take corrective action when errors are identified. Corrective action may include executing a work around process and then identifying the cause and solution for data errors.
• Participate as a full member of the global Analytics team, providing solutions for and insights into data related items.
• Collaborate with your Data Science and Business Intelligence colleagues across the world to share key learnings, leverage ideas and solutions and to propagate best practices.
• You will lead projects that include other team members and participate in projects led by other team members.
• Apply change management tools including training, communication and documentation to manage upgrades, changes and data migrations.

Must Have Skills:

• Azure Databricks
• Azure Data Factory
• PySpark
• Spark - SQL
• ADLS

Job Type: Contractual / Temporary
Contract length: 6 months

Pay: Up to ₹2,000,000.00 per year

Schedule:

• Monday to Friday

Experience:

• Databricks: 3 years (Required)
• Azure data factory: 3 years (Required)
• Pyspark: 3 years (Required)

Work Location: In person