Data Engineer

Overview

• Setup and configuration of metadata management tools for optimum performance.
• Scan and catalog data assets, including creation of data lineage.
• Coordinate with the vendors to resolve tool issues and to request enhancements.
• Integration between the tools to provide single pane of glass for business users.
• Perform data profiling and automate the jobs.
• Develop and implement Data Quality rules.
• Design and build solutions using the tools for stakeholder requirements.
• Support users with data uploads and updates using tool templates.
• Setup workflows for Data Maintenance.
• Monitor and maintain data quality metrics and dashboards.
• Create and maintain technical documentation for metadata management processes.
• Collaborate with data engineering teams to ensure data pipeline quality.
• Participate in data governance initiatives and policy implementation.
• Provide training and support to end users on metadata and data quality tools.
  
  

• Minimum engineering degree in computer science or related field
• Minimum 3 years experience in Data Quality using Informatica IDQ/BDQ or other Data Observability platforms like Anomalo.
• Minimum 2 years experience with Alation Data Governance/Data Catalog tool. Experience on Informatica AXON/EDC will also be considered.
• Good knowledge on SQL to be able to query data from databases.
• Some development experience preferred with Python language to be able to automate jobs/tasks.
• Ability to quickly learn new tools and technologies in a dynamic environment.
• Understanding of data governance principles and data management best practices.
• Experience with API-based integrations and REST APIs for tool connectivity (preferred).
• Knowledge of AWS cloud platform and their services (preferred).
• Familiarity with Databricks platform and Databricks SQL for data engineering workflows (preferred).
• Experience with version control systems (Git) and modern software development practices.
• Strong analytical and problem-solving skills with attention to detail.
• Ability to work independently and collaboratively in a team environment.
• Able to provide out of the box solutions.
• Good communication skills, both verbal and written.