Lead Data Engineer

Overview

As a member of the Platform engineering team, you will be the key techno functional expert leading and overseeing PepsiCo's Platforms & operations and drive a strong vision for how Platforms engineering can proactively create a positive impact on the business. You'll be an empowered Leader of a team of Platform engineers who build Platform products for platform optimization and cost optimization and build tools for Platform ops and Data Ops on the PepsiCo Data Lake, and enable exploration and access for analytics, visualization, machine learning, and product development efforts across the company. As Leader of the Platform engineering team, you will help in managing platform Governance team that builds frameworks to guardrail the platforms of very large and complex data applications in public cloud environments and directly impact the design, architecture, and implementation of PepsiCo's flagship data products around topics like revenue management, supply chain, manufacturing, and logistics. You will work closely with process owners, product owners and business users. You'll be working in a hybrid environment with in-house, on-premises data sources as well as cloud and remote systems.

Responsibilities

• Active contributor to cost optimization of platforms and services.
• Manage and scale Azure Data Platforms to support new product launches and drive Platform Stability and Observability across data products.
• Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for Data Platforms for cost and performance.
• Responsible for implementing best practices around systems integration, security, performance and Platform management.
• Empower the business by creating value through the increased adoption of data, data science and business intelligence landscape.
• Collaborate with internal clients (data science and product teams) to drive solutioning and POC discussions.
• Evolve the architectural capabilities and maturity of the data platform by engaging with enterprise architects and strategic internal and external partners.
• Develop and optimize procedures to “production Alize” data science models.
• Define and manage SLAs for Platforms and processes running in production.
• Support large-scale experimentation done by data scientists.
• Prototype new approaches and build solutions at scale.
• Research in state-of-the-art methodologies.
• Create documentation for learnings and knowledge transfer.
• Create and audit reusable packages or libraries.
  
  

Qualifications

• 8+ years of overall technology experience that includes at least 4+ years of hands-on software development, Program management, and data engineering
• 4+ years of experience with Data Lake Infrastructure, Data Warehousing, and Data Analytics tools.
• 4+ years of experience in Databricks optimization and performance tuning
• Experience in managing multiple teams and coordinating with different stakeholders to implement the vision of the team.
• Fluent with Azure cloud services. Azure Certification is a plus.
• Experience with integration of multi cloud services with on-premises technologies.
• Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines.
• Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets.
• Experience with at least one MPP database technology such as Redshift, Synapse or SnowFlake.
• Experience with version control systems like Github and deployment & CI tools.
• Experience with Azure Data Factory, Azure Databricks.
• Experience with Statistical/ML techniques is a plus.
• Experience with building solutions in the retail or in the supply chain space is a plus
• Understanding of metadata management, data lineage, and data glossaries is a plus.
• Working knowledge of agile development, including DevOps and DataOps concepts.
• Familiarity with business intelligence tools (such as PowerBI).