Web Crawler Developer

Overview

Job Description:

We are seeking a creative and independent Web Crawler Developer to join our team Seattle based Construction Team. The ideal candidate will have a keen eye for detail, a passion for problem-solving, and the ability to think outside the box to develop sophisticated web scraping solutions.

Responsibilities:

- Design, implement, and maintain web crawlers that can effectively extract data from various websites .

- Analyze web page structures and adapt crawlers to extract relevant information efficiently.

- Monitor crawler performance and make necessary adjustments to ensure optimal data collection.

- Work independently to identify new opportunities for data extraction and offer insightful recommendations.

- Ensure compliance with legal and ethical standards for data scraping.

- Collaborate with data analysts and other team members to understand data needs and improve data accuracy.

- Keep up-to-date with the latest web scraping technologies and best practices

Qualifications:

- Strong experience with web scraping tools and frameworks (e.g., Scrapy, BeautifulSoup, Selenium, etc.).

- Proficiency in programming languages such as Python, Java, or others relevant to web crawling.

- Experience with handling and parsing different data formats like HTML, JSON, XML, etc.

- Excellent problem-solving skills and the ability to think outside the box.

- Ability to work independently and manage multiple tasks efficiently.

- Solid understanding of web protocols (HTTP, HTTPS) and web technologies.

- Familiarity with version control systems, preferably Git.

- Knowledge of data privacy laws and ethical web scraping practices.

Preferred:

- Experience with cloud services like AWS or Azure for deploying and managing web crawlers.

- Understanding of databases and data storage solutions.

- Previous experience in a similar role or related projects.

Job Type: Contractual / Temporary
Contract length: 2 months

Pay: ₹76,000.00 - ₹80,000.00 per month

Benefits:

• Work from home

Supplemental Pay:

• Performance bonus

Expected Start Date: 03/06/2025