Software Engineer - Data Collection System

Overview

We are seeking a highly motivated and technically skilled Software Engineer to join our Data Collection team.

You will be responsible for designing, developing, and maintaining high-performance systems that extract and process data from a diverse range of online sources.

This role is ideal for someone who enjoys solving complex engineering challenges, writing production ready code, exploring new technologies, and continuously improving data collection processes.

Key Responsibilities

• Design, build, and maintain scalable and reliable data collection and processing systems.
• Write clean, efficient, and maintainable code to handle large-scale data extraction across varied sources.
• Debug and resolve issues related to data quality, accuracy, and performance.
• Collaborate with data engineering, infrastructure, and product teams to meet evolving data requirements.
• Continuously enhance system performance, tooling, automation, and developer workflows.
• Maintain comprehensive documentation and ensure strong version control practices.
• Ensure all data extraction processes adhere to legal, ethical, and compliance guidelines.
  
  

• 4-6 years of hands-on software engineering experience.
• Proficiency in Scala or Java (experience with Python or Rust is a strong plus).
• Strong knowledge of distributed systems, networking, and data processing pipelines.
• Hands-on experience with major cloud platforms (AWS/GCP/Azure) and Docker.
• Familiarity with CI/CD systems, workflow schedulers (Airflow or similar), and Git version control.
• Experience working with various data storage technologies - SQL, NoSQL, S3, etc.
• Excellent problem-solving and debugging abilities.
• Effective communication and collaboration skills with a proactive approach to knowledge sharing.
• Demonstrated growth mindset with a passion for learning, experimentation, and continual improvement
  
  

(ref:hirist.tech)