Sr. Data Engineer - Python

Description

• Develop, test, and maintain scalable Python-based applications and data pipelines
• Design and implement ETL/ELT processes for structured and unstructured data
• Work with large-scale data processing frameworks (e.g., Spark, Hadoop)
• Build and optimize data architectures and data models
• Collaborate with data analysts, data scientists, and cross-functional teams
• Ensure data quality, integrity, and security across systems
• Integrate data from multiple sources including APIs, databases, and third-party tools
• Monitor and troubleshoot data workflows and performance issues
• Contribute to system design and architecture discussions

• Strong proficiency in Python
• Experience with data engineering concepts and tools
• Hands-on experience with SQL and relational databases (e.g., PostgreSQL, MySQL)
• Experience with big data technologies like Apache Spark / Hadoop
• Familiarity with workflow orchestration tools (e.g., Airflow)
• Experience with cloud platforms such as AWS / Azure / GCP
• Knowledge of S3, EMR, Glue or similar services (preferred for AWS environments)
• Understanding of REST APIs and data integration techniques
• Strong problem-solving and analytical skills

• Experience with data warehousing solutions (e.g., Redshift, Snowflake, BigQuery)
• Familiarity with containerization tools like Docker and Kubernetes
• Exposure to CI/CD pipelines
• Knowledge of data governance and security best practices

• Strong communication and collaboration skills
• Ability to work in a fast-paced, agile environment
• Self-driven with a proactive mindset