Data Engineer

About The Role

A data engineer designs, constructs, installs, tests, and maintains highly scalable data management systems. They are responsible for building the infrastructure that allows for the generation, collection, and analysis of large datasets. Key responsibilities include developing, constructing, testing, and maintaining architectures such as databases and large-scale processing systems, ensuring that architectures support data analytics, and preparing data for prescriptive and predictive modeling. Data engineers also develop data set processes for data modeling, mining, and production, integrate new data management technologies and software engineering tools into existing structures, and collaborate with data scientists and analysts to ensure data accuracy and accessibility. They play a critical role in enabling the data-driven decision-making process by ensuring that data pipelines are robust, efficient, and scalable

What You Will Deliver

• Part of a cross-disciplinary team, working closely with other data engineers, software engineers, data scientists, data managers and business partners.
• Implements and maintains reliable and scalable data infrastructure to move, process and serve data.
• Writes, deploys and maintains software to build, integrate, manage, maintain, and quality-assure data at bp.
• Adheres to and advocates for software engineering best practices (e.g. technical design, technical design review, unit testing, monitoring & alerting, checking in code, code review, documentation), code reuse).
• Adheres to and advocates for data engineering best practices(e.g. data modeling, pipeline idempotency, operational observability)
• Responsible for deploying secure and well-tested software and data-assets that meet privacy and compliance requirements; develops, maintains and improves CI / CD pipeline,
• Responsible for service reliability and following site-reliability engineering best practices: on-call rotations for services they maintain, responsible for defining and maintaining SLAs. Help design, build, deploy and maintain infrastructure as code. Containerizes server deployments.
• Actively contributes to improve developer velocity.
  
  

What you will need to be successful (experience and qualifications)

Essential

• Hands-on experience designing, planning, building, productionizing, maintaining and documenting reliable and scalable data infrastructure and data products in complex environments
• Familiarity with common energy industry data structures, formats, platforms and protocols.
• Development experience in one or more object-oriented programming languages (e.g. Python, Scala, Java, C#)
• Experience with SQL and noSQL database fundamentals, query structures and design best practices, including scalability, readability, and reliability
• Experience implementing large-scale distributed systems in collaboration with more senior team members
• Knowledge and hands-on experience in technologies across all data lifecycle stages
• Strong verbal and written communication skills
• Continuous learning and improvement mindset
• BS degree in computer science or related field or equivalent knowledge and experience
  
  

Desired

• Previous experience of Data Engineering within either the energy sector or adjacent industries