Data Engineer

Overview

Profile: Data Engineer

Skills: PySpark, Hadoop , Teradata, Python

Experience - 7+ Years

Time- 3-4 Hrs Per Day (Between 7 AM IST to 11 AM IST)

Location: Remote

Job Description:

We are looking for a skilled Data Engineer to join our data platform team. The ideal candidate will be responsible for designing, developing, and maintaining robust data pipelines using PySpark, Hadoop, Teradata, and Python. You will work closely with data scientists, analysts, and business stakeholders to ensure high data quality and accessibility.

Key Responsibilities:

• Design and build scalable data pipelines using PySpark and Hadoop ecosystem.
• Perform data extraction, transformation, and loading (ETL) from various sources including Teradata.
• Optimize performance of data workflows on large datasets.
• Collaborate with data scientists and analysts to support advanced analytics and reporting needs.
• Ensure data accuracy, consistency, and security across systems.
• Monitor job performance and troubleshoot pipeline failures or latency issues.

Qualifications:

• Bachelor’s or Master’s in Computer Science, Information Technology, or related field.
• Strong experience with PySpark and distributed data processing.
• Proficiency in Hadoop components like HDFS, Hive, and YARN.
• Solid working knowledge of Teradata (SQL, BTEQ scripts, etc.).
• Advanced Python scripting skills for data manipulation and automation.
• Familiarity with version control (Git) and workflow orchestration tools (e.g., Airflow is a plus).
• Strong analytical and problem-solving skills.

Nice to Have:

• Experience with cloud platforms (AWS, Azure, or GCP).
• Knowledge of Delta Lake, Databricks, or Apache Kafka.
• CI/CD experience for data pipelines.

Job Types: Part-time, Contractual / Temporary, Freelance
Contract length: 6 months

Pay: From ₹28,000.00 per month

Expected hours: 15 – 20 per week

Benefits:

• Work from home

Experience:

• Data Engineer: 7 years (Preferred)
• Hadoop: 4 years (Preferred)
• Teradata: 4 years (Preferred)
• Python: 4 years (Preferred)

Work Location: Remote