Senior Data Engineer (Databricks, Azure/AWS, Python/R/PySpark)

Our Purpose

Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.

Title And Summary

Senior Data Engineer (Databricks, Azure/AWS, Python/R/PySpark)

Overview

The Data & Analytics team is looking for a Databricks Platform Engineer to drive our mission to unlock potential of data assets on AWS cloud by consistently innovating, and moving data to our Cloud platforms and making our business engineering team consume it with the set standards and principles. Our Cloud Data Platform (Databricks) is being used by various groups to derive insights, perform Machine Learning and Data Science activities which in turn supports various revenue generating initiatives across the organization.

Role - Databricks Platform Engineer

• Design and develop high quality, secure, scalable software solutions based on technical requirements specifications and design artifacts within expected time and budget
• Develop and build solutions on cloud data platforms - Databricks - adhering to industry standards.
• Develop and build Databricks as a platform for the engineering community to use
• Build governance framework for cloud data storage and usage for enterprise data platforms.
• Collaborate on cross-functional agile teams - that include Developers, Report/BI Developers and Product Owners
• Work closely with Product Owners to understand cloud use cases and determine the best way to implement those.
• Provide technology leadership in the cloud big data space to development and product teams.
• Provide technology leadership to other leaders in big data and cloud data platform space.
• Stay abreast of Data Platform technology trends and industry best practices to hone and maintain your talent
• Participate in architectural discussions , iteration planning, and feature sizing meetings
• Adhere to Agile processes and participate actively in agile ceremonies
• Stakeholder management skills
  
  
  

All About You

• 6-10 years of hands on experience with Cloud data platform
• Hands on experience in Databricks and deep understanding of its architecture
• Experience managing, developing and supporting data lakes, lakehouses, warehouse on premise and on cloud, ETL solutions and other analytics solutions. Exposure to technology trends like R, Python and similar technology is added advantage.
• Experience working with development/engineering teams in a global environment
• In depth practical experience on native cloud services - Azure and/or AWS.
• Strong understanding of cloud DevOps practices and implementation.
• Experience in cloud data migration for large sized enterprise data warehouse environments.
• Understanding of cloud data governance include compute optimization, cost control, user profile management etc.
• In depth understanding of cloud operations strategy and user management across various cloud providers and cloud platforms.
• Strong understanding and hands-on on cloud security models, encryption strategy, network layers for on-prem + cloud hybrid model and other related concepts.
• Experience in establishing new engineering team and working towards taking the team to steady state
• Understanding of database technologies like Exadata/Oracle/Netezza/SQL server databases
• Understanding of SDLC and experience in establishing processes, standards and governance to bring efficiency within development team
  
  
  

Corporate Security Responsibility

All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:

• Abide by Mastercard’s security policies and practices;
• Ensure the confidentiality and integrity of the information being accessed;
• Report any suspected information security violation or breach, and
• Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.