Data Engineer with Golang/Python Skills

Overview

About Us
Our mission is to help brands optimize stock management, boost sales, and gain competitive insights through real-time data tools. We're looking for a skilled Data Engineer to join us and scale our WhatsApp stock alert system and analytics platform.

Role Overview
As our Data Engineer, you'll design, build, and maintain the data pipelines that power our real-time stock monitoring, competitor analysis, and market trend features.

Key Responsibilities

Data Pipeline Development: Build scalable ETL/ELT pipelines to ingest, process, and store data from quick commerce platforms (APIs, web scraping, partner feeds)

Real-time Processing: Implement streaming pipelines for WhatsApp stock-out alerts using Kafka, Apache Flink, or similar technologies

Data Architecture: Design data warehouse/lake solutions (Snowflake, BigQuery, or AWS Redshift) optimized for analytics and ML workloads

API Integration: Connect with e-commerce APIs, competitor data sources, and third-party services

Monitoring & Alerts: Develop systems for real-time stock availability monitoring and automated notifications

Data Quality: Implement data validation, monitoring, and cleansing processes

Performance Optimization: Ensure low-latency queries and efficient data processing for real-time dashboards

Collaboration: Work with developers to integrate data pipelines into our application stack