Associate Data Engineer

Overview

Introduction:
Our goal at Pivotree is to help accelerate the future of frictionless commerce. We will help lead this change over the next decade because we believe a future where technology is embedded intimately into all aspects of our everyday lives can benefit everyone and will shape the interactions with the brands we love. We will help shape the future of frictionless commerce by working together with some of the best brands in the world and some of the best people in the industry to leverage converging technologies that will make it possible to accelerate frictionless commerce faster than ever.

Pivotree provides services focused on the design, implementation, management, and maintenance of complex ecommerce solutions for large enterprises. We provide the technical skills necessary to enable the effective use of technologies combined with the business context to leverage a solution to solve our clients' business challenges. We strive to fill the gaps in available technology with our own IP to reduce the barriers to adoption.

We enable inclusive, immersive and highly personalized experiences for our clients and their customers. We build our products with a view to productizing and scaling technology to lower the costs and reduce the risks of implementing and managing our integrated solutions. Each of our solutions starts with reliable and reputable e-commerce and MDM platforms, which run on enterprise grade infrastructure that are customized to meet a variety of client needs, situations, and budgets. Over the next 10 years we will add new categories and capabilities that will define frictionless commerce ecosystems.

This is a journey of technology acceleration combined with consumer readiness and adoption. We are looking for people capable of adapting relentlessly to the rapidly evolving world around us.

Job Overview:
We are looking for a Associate Data Engineer (1-3 years’ experience) to develop, maintain, and
optimize web data extraction processes. In this role, you will work with Python, SQL, Excel and VBA
to automate web scraping workflows and extract valuable data from online sources. You will also
help ensure data accuracy, troubleshoot scraping challenges, and support integration with databases
and reporting tools. This role is ideal for someone with a foundational understanding of web
scraping and data automation who is looking to enhance their skills and take on more
responsibilities.

Key Responsibilities:1. Web Scraping & Data Extraction:
o Develop and maintain Python-based web scraping scripts to extract structured data
from websites.
o Work with Scrapy, BeautifulSoup, or Selenium to handle dynamic and static
websites.
o Assist in handling basic anti-scraping techniques, such as user-agent rotation and
proxy usage.

2. Data Processing & Transformation:
o Clean and structure scraped data for analysis and storage.
o Use Excel, VBA, and SQL queries for basic data transformation and manipulation.
o Identify and correct data inconsistencies in extracted datasets.

3. Scripting & Automation:
o Write and maintain Python and VBA scripts for data processing and automation.
o Automate repetitive scraping and data handling tasks for efficiency.

4. ETL & Data Integration:
o Assist in ETL (Extract, Transform, Load) processes for scraped data.
o Support integration of scraped data into SQL databases or Excel reports.
o Monitor scraping scripts and workflows for errors and necessary updates.

5. Data Quality & Governance:
o Perform basic data validation and quality checks on extracted data.
o Update scraping workflows when websites change structures.
o Ensure ethical web scraping practices and compliance with guidelines.

6. Reporting & Visualization:
o Create basic reports and summaries using Excel and SQL queries.
o Support automation of data reports and scheduled updates.

Qualifications & Skills:

• 1-3 years of experience in web scraping, data automation, or ETL.
• Basic to intermediate proficiency in Python (Scrapy, BeautifulSoup, Selenium) for web

scraping.

• Experience with SQL, Excel, and VBA for data manipulation and reporting.
• Understanding of HTML, CSS, and JavaScript to analyze website structures.
• Strong problem-solving and analytical skills with attention to data accuracy.

Preferred Skills:

• Basic knowledge of API integrations and database management.
• Familiarity with cloud storage solutions (AWS, Azure, Google Cloud).
• Exposure to anti-scraping measures and basic workarounds (proxies, headers, delays).

If you are passionate about web data extraction, automation, and analytics, we encourage you to
apply! Join us and enhance your skills in web scraping and data automation.

Pivotree is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive and accessible workplace.