Senior Data Engineer

Job Description

Job Purpose

ICE Data Services, a subsidiary of Intercontinental Exchange, presents a unique opportunity for a full-time Senior Data Engineer within the Climate Transition Risk Group to join a team responsible for creating customised & innovative product and data solutions for clients & third-party partner firms which use ICE Data Services Climate Transition Risk products and data solutions.

The role is working in the Climate Transition Finance team working with emissions data, financials data, and other transition risk datasets. The role will include building data pipelines for new datasets, maintaining and updating databases, and building high quality data dashboards to visualise datasets. The role will involve analysing complex datasets effectively in a variety of settings and presenting the findings to the wider team. The role will also include some aspects of AI/ML and data scraping with NLP.

The role requires a candidate that can both apply and expand their existing technical skillset and a keenness to learn about climate risk datasets and contribute to the advancement of our products. The candidate is expected to work independently with minimal supervision and can manage multiple projects and priorities with demanding deadlines.

Domain knowledge in climate/carbon emissions data or financial datasets is preferred but not essential.

Responsibilities

• Implementation, optimization, and maintenance of our data pipelines, ensuring they are scalable, secure, and reliable for processing vast amounts of legal data.
• Architect and implement solutions involving Large Language Models (LLMs) and Natural Language Processing (NLP) to extract insights and create advanced features for our analytics platforms.
• Collaborate closely with product, engineering, and domain experts to build intelligent data systems that enhance the user experience.
• Write quality code, and ability to adhere to the coding guidelines.
• Pursue opportunities to improve delivery quality and efficiency within the organization.
• Follow, learn and apply new big data technologies and innovations.
• Ensure data accuracy and integrity across multiple sources and systems.
  
  

Knowledge And Experience

• 5+ years of experience in data engineering within a production environment.
• Experience with SQL/NoSQL databases, preferably MongoDB (Medium level 4+ years).
• Strong programming skills in languages such as Python, and experience with data processing frameworks like Apache Spark, Airflow, or Kafka. (High level, 6-7+ years).
• Experience building interactive dashboards with Streamlit or Plotly Dash (Medium level 2+ years).
• Familiarity with cloud platforms (AWS, GCP, Azure) and containerization technologies like Docker and Kubernetes.