Microsoft Azure Data Engineer APAC

Job Title

Microsoft Azure Data Engineer APAC

Job Description Summary

We are currently seeking a talented Data Engineer who will design, develop, and maintains data solutions across APAC Advisory products. The Data Engineer will be responsible for ensuring data quality, security, scalability, and performance of the data products and collaborate with product managers, developers, analysts, and stakeholders to understand the business requirements and translate them into data models and architectures.

This role offers an exciting opportunity to partner with other Data Architect, Engineers, Data Scientists within TDS in addition to our business stakeholders across the APAC Business to support C&W with development and maintenance of multiple Data products to make a significant impact on product effectiveness and user experiences, ensuring the organization’s data assets are effectively leveraged for decision-making, analytics, and operational efficiency.

Job Description

As a Microsoft Azure Data Engineer, you will be responsible for designing, implementing, and managing data pipelines and architectures on the Azure Synapse platform. You will work with a variety of data sources and ensure data is processed and stored efficiently using the Medallion Architecture (Bronze, Silver, Gold layers). Your expertise in PySpark, Synapse pipelines, API integrations, Azure Synaspse Analytics and integration with Master Data Management tools like Profisee will be crucial in developing robust data solutions that meet our business needs.

Key Responsibilities:

• Design and implement data pipelines using Pyspark Notebooks.
• Manage and optimize data storage and processing through the Medallion Architecture (Bronze, Silver, Gold layers).
• Develop and maintain ETL processes using notebooks to ingest, transform, and load data from various sources.
• Integrate APIs to facilitate data exchange between systems.
• Create and manage table views to surface data to end users.
• Ensure data quality, consistency, and security across all layers.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Monitor and troubleshoot data pipelines to ensure reliability and performance.
• Stay up to date with the latest Azure technologies and best practices.

Qualifications:

• Bachelor’s degree in computer science, Information Technology, or a related field.
• Proven experience as a Data Engineer with a focus on Azure technologies.
• Strong proficiency in PySpark notebooks and Synapse pipelines with at least 2 years of demonstrable experience with notebooks
• Experience with Medallion Architecture and managing data through bronze, silver, and gold layers.
• Familiarity with API integration and data exchange.
• Experience with master data management tools such as Profisee.
• Ability to create and manage table views for data visualization.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.
• Ability to work in a fast-paced, dynamic environment.

Preferred Qualifications:

• Azure certifications (e.g., Azure Data Engineer Associate).
• Experience with other Azure services such as Azure Databricks, Azure Synapse Analytics, and Azure SQL Database.
• Knowledge of data governance and security best practices.
• Experience with additional master data management tools and techniques.

INCO: “Cushman & Wakefield”