Data Acquisition & Web Application Developer

Experience: 3.00 + years

Salary: USD 1851-2962 / month (based on experience)

Expected Notice Period: 15 Days

Shift: (GMT+05:30) Asia/Kolkata (IST)

Opportunity Type: Remote

Placement Type: Full Time Contract for 3 Months(40 hrs a week/160 hrs a month)

(*Note: This is a requirement for one of Uplers' client - GPRO Ltd)

What do you need for this opportunity?

Must have skills required:

Analytics, Monitoring, stream processing, Web application deployment, APIS, data acquisition, Web scraping, Agile, Node Js, Python

GPRO Ltd is Looking for:

Job Title: Data Acquisition & Web Application Developer

About the Project:

We are seeking a skilled full-stack developer to build a specialised web application designed to aggregate and present public information on individuals, such as company executives and leaders. This tool will serve as a comprehensive profile generator, pulling data from diverse online sources including news outlets, social media and other platforms. The primary goal is to provide users with a centralised, easily navigable view of a person's online presence, latest news and public information.

Project Overview:

The core of this project involves developing a robust data acquisition layer capable of scraping and integrating information from various online sources. This data will then be presented through a user-friendly web interface. The application should allow users to input

a person's name and receive an aggregated view of relevant public data.

Key Responsibilities:

• Develop and Implement Data Acquisition Layer: Design and build systems to scrape and collect data from specified sources, including news websites (e.g., Bloomberg.com, Reuters, BBC.com, Financial Times), social media (e.g., X, LinkedIn), and media platforms (e.g., YouTube, podcasts).
• Integrate with APIs: Utilize official APIs (e.g., Bloomberg data, Reuters, FinancialTimes, Google Finance) where available and prioritized. Evaluate and integrate with third-party scraping APIs (e.g., Apify, Oxylabs, SerpApi) as necessary, considering associated risks and subscription models.
• Handle Hybrid Approach: Implement a strategy that leverages licensed APIs for premium sources while potentially using third-party scrapers for others, being mindful of terms of service and legal/ethical considerations. Direct scraping of highly protected sites like Bloomberg, Reuters, and FT should be avoided or approached with extreme caution using third-party services.
• Design Data Storage and Indexing: Determine appropriate data storage solutions, considering the volume of data and its relevance over time. Implement indexing and caching mechanisms to ensure efficient search and retrieval of information, supporting near real-time data presentation.
• Develop Web Application Front-End: Build a basic, functional front-end interface similar to the provided examples ("Opening Screen," "Person profile"). This includes displaying the aggregated information clearly.
• Implement User Functionality: Enable users to:
  
  
  

○ Input a person's name for searching.

○ Sort displayed outputs by date.

○ Click through links to access the original source of information.

○ Navigate to a new search easily (e.g., via a tab).

• Consider Stream Processing: Evaluate and potentially implement stream processing techniques for handling near real-time data acquisition and updates.
• Ensure Scalability: Design the application to support a specified level of concurrent searches (estimated at 200 for the initial phase).
• Build Business Informational Layer: Develop a component that tracks the usage of different data services (APIs, scrapers) for monitoring costs and informing future scaling decisions.
• Technical Documentation: Provide clear documentation for the developed system, including data flows, API integrations, and deployment notes.
  
  
  

Required Skills and Experience:

• Proven experience in web scraping and data acquisition from diverse online sources.
• Strong proficiency in developing with APIs, including handling different authentication methods and data formats.
• Experience with relevant programming languages and frameworks for web development and data processing (e.g., Python, Node.js, etc.).
• Knowledge of database design and data storage solutions.
• Familiarity with indexing and caching strategies for search applications.
• Understanding of potential challenges in web scraping (e.g., anti-scraping measures, terms of service).
• Experience in building basic web application front-ends.
• Ability to consider scalability and performance in system design.
• Strong problem-solving skills and ability to work independently or as part of a small team.
• Experience working with foreign (western based) startups and clients. Ability to work in agile environments and ability to pivot fast.
  
  
  

Desirable Skills:

• Experience with stream processing technologies.
• Familiarity with deploying and managing web applications (though infrastructure design is flexible).
• Experience with monitoring and analytics for application usage.
  
  
  

How to apply for this opportunity?

• Step 1: Click On Apply! And Register or Login on our portal.
• Step 2: Complete the Screening Form & Upload updated Resume
• Step 3: Increase your chances to get shortlisted & meet the client for the Interview!
  
  
  

About Uplers:

Our goal is to make hiring reliable, simple, and fast. Our role will be to help all our talents find and apply for relevant contractual onsite opportunities and progress in their career. We will support any grievances or challenges you may face during the engagement.

(Note: There are many more opportunities apart from this on the portal. Depending on the assessments you clear, you can apply for them as well).

So, if you are ready for a new challenge, a great work environment, and an opportunity to take your career to the next level, don't hesitate to apply today. We are waiting for you!