Senior Data Engineer

THE ROLE

• As a Senior Data Engineer, you will be part of a team who design, build and maintain scalable, secure, and high-performance data solutions on Microsoft Azure. You will support mission-critical operations across oil trading, refining, and retail fuel distribution by building robust data pipelines, integrating real-time telemetry, and ensuring compliance with regional and international regulations.
• This role sits within the Oil Assets IT team, reporting to the Data Engineering Team Lead, with occasional travel to affiliates in South Africa and Brazil.
  
  

PROFILE

• Seasoned data engineer with 7+ years of experience, and a strong foundation in designing and managing cloud-native data pipelines, data models, and analytics solutions. Brings hands-on expertise in Azure-based technologies including Synapse Analytics, Data Factory, and Azure SQL, with forward-looking experience in Microsoft Fabric and Databricks. Operates in a fast-paced, business-facing environment, delivering high-quality solutions across multiple parallel workstreams. Self-driven and adaptable, with a deep understanding of the downstream oil and gas domain and a commitment to continuous learning and innovation.
  
  

Responsibilities

• Design and manage scalable data pipelines and ETL/ELT processes using Azure Data Factory, Synapse Analytics, and emerging platforms such as Microsoft Fabric and Databricks.
• Collaborate with peers to create and maintain data models and databases in Azure SQL DB and Azure Data Lake.
• Ensure data quality, lineage, and availability through rigorous validation, testing, and monitoring practices.
• Integrate data from ETRM systems, business applications, refinery control systems, and retail station point-of-sale networks.
• Implement real-time analytics solutions using Azure Event Hubs, Stream Analytics, and IoT Hub.
• Liaise with multiple functional groups across Glencore Group, the Oil Department and the industrial assets, to provision and deploy infrastructure within Azure.
• Implement data governance and security policies using Microsoft Purview and Azure RBAC.
• Ensure data quality, lineage, and availability for business-critical applications.
• Deliver high-velocity solutions supported by strong coding practices and automation.
• Build and maintain multiple pipelines in parallel, managing context switching effectively.
• Stay current with modern data technologies and trends, including AI/ML, and advise on their responsible adoption.
• Operate independently, driving individual workstreams while contributing to team-wide initiatives
  
  

COMPETENCIES

Core Competencies

• Bachelor’s degree in Computer Science, Information Technology, or equivalent experience.
• Data Engineering and Modeling: Strong grasp of core data modeling concepts and techniques, experience designing and managing ETL/ELT pipelines using Azure Data Factory and Synapse Analytics.
• Programming Languages: Expert proficiency in SQL, Python, and PySpark for data transformation, validation, and analytics.
• Cloud Technologies (Azure):
• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Azure Databricks
• Azure Data Lake Storage Gen2
• Azure SQL
• Azure Functions
• Version Control and CI/CD: Experience with Git/GitHub and CI/CD pipelines using GitHub Actions.
  
  

Desirable

• Visualisation and Reporting: Skilled in building dashboards and visual narratives using Power BI.
• Infrastructure and Automation: Familiarity with infrastructure-as-code tools such as Terraform; experience provisioning resources in Azure.
• Data Governance and Security: Experience implementing data lineage, cataloguing, and access controls using Microsoft Purview and Azure RBAC.
• Real-Time and Streaming Data: Exposure to Azure Event Hubs, IoT Hub, and Azure Stream Analytics for real-time data processing.
• Advanced Tools and Platforms:
• Apache Airflow for workflow orchestration
• Microsoft Power Platform for low-code solutions
• Azure DevOps for collaborative development and deployment
• NoSQL/Big Data technologies (e.g., Cosmos DB, Hadoop, or similar)
• Machine Learning Engineering: Understanding of ML workflows and integration into data pipelines.
  
  

Certifications

• Microsoft Certified: Azure Data Engineer Associate
• Microsoft Certified: Azure Solutions Architect Expert
• Microsoft Certified: Azure Fundamentals (AZ-900)
• Microsoft Certified: Azure AI Engineer Associate
  
  

Apache Spark Developer Certific

Who We Are

Glencore is one of the world’s largest global diversified natural resource companies and a major producer and marketer of more than 60 commodities that advance everyday life. Through a network of assets, customers and suppliers that spans the globe, we produce, process, recycle, source, market and distribute the commodities that support decarbonisation while meeting the energy needs of today.

With over 150,000 employees and contractors and a strong footprint in over 35 countries in both established and emerging regions for natural resources, our marketing and industrial activities are supported by a global network of more than 50 offices.

Glencore’s customers are industrial consumers, such as those in the automotive, steel, power generation, battery manufacturing and oil sectors. We also provide financing, logistics and other services to producers and consumers of commodities.

Glencore is proud to be a member of the Voluntary Principles on Security and Human Rights and the International Council on Mining and Metals. We are an active participant in the Extractive Industries Transparency Initiative.

We will support the global effort to achieve the goals of the Paris Agreement through our efforts to decarbonise our own operational footprint. We believe that we should take a holistic approach and have considered our commitment through the lens of our global industrial emissions. Against a restated 2019 baseline, we are targeting to reduce our Scope 1, 2 and 3 industrial emissions by 15% by the end of 2026, 25% by the end of 2030, 50% by the end of 2035 and we have an ambition to achieve net zero industrial emissions by the end of 2050, subject to a supportive policy environment. For more information see our 2024-2026 Climate Action Transition Plan and the About our emissions calculation and reporting section in our 2023 Annual Report, available on our website at glencore.com/publications.

Note*: All applications will be subject to potential background checks. You will be informed if this will be the case. Further information about the processing of your personal data will be made available during the application process. We'll seek your explicit consent prior to processing the data.