Python Data Engineer

Proficiency in Python, especially for data extraction and automation tasks.
Strong experience with web scraping frameworks such as Scrapy, BeautifulSoup, or Selenium.
Hands-on experience building automated data pipelines using tools like Airflow, Luigi, or custom schedulers.
Knowledge of web data collection techniques, including handling pagination, AJAX, JavaScript-rendered content, and rate-limiting.
Familiarity with RESTful APIs and techniques for API-based data ingestion.
Experience with data storage solutions, such as PostgreSQL, MongoDB, or cloud-based storage (e.g., AWS S3, Google Cloud Storage).
Version control proficiency, especially with Git.
Ability to write clean, modular, and well-documented code.
Strong debugging and problem-solving skills in data acquisition workflows.
Nice-to-Haves
Experience with cloud platforms (AWS, GCP, or Azure) for deploying and managing data pipelines.
Familiarity with containerization tools like Docker.
Knowledge of data quality monitoring and validation techniques.
Exposure to data transformation tools (e.g., dbt).
Understanding of ethical and legal considerations in web scraping.
Experience working with CI/CD pipelines for data workflows.
Familiarity with data visualization tools (e.g., Tableau, Power BI, or Plotly) for quick insights.
Background in data science or analytics to support downstream use cases.