Data Engineer

Overview

We are looking for an experienced Data Engineer with 2-4 years of hands-on experience in building scalable data pipelines and processing large datasets. The ideal candidate must have strong expertise in PySpark and exposure to real-time or streaming frameworks such as Apache FlinkYou will work closely with architects, data scientists, and product teams to design and deliver robust, high-performance data solutions.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using PySpark.
• Implement real-time or near-real-time data processing using Apache Flink.
• Optimize data workflows for performance, scalability, and reliability.
• Work with large-scale data platforms and distributed environments.
• Collaborate with cross-functional teams to integrate data solutions into products and analytics platforms.
• Ensure data quality, integrity, and governance across pipelines.
• Conduct performance tuning, debugging, and root-cause analysis of data processes.
• Write clean, modular, and well-documented code following best engineering practices.
  
  

• Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
  
  

• Strong hands-on experience in PySpark(RDD, DataFrame API, Spark SQL).
• Experience with Apache Flink (streaming or batch).
• Solid understanding of distributed computing concepts.
• Proficiency in Python for data engineering workflows.
• Strong SQL skills for data manipulation and transformation.
• Experience with data pipeline orchestration tools (Airflow, Step Functions, etc. ).
  
  

• Experience with cloud platforms (AWS / Azure / GCP).
• Knowledge of data lakes, lakehouse architectures, and modern data stack tools.
• Familiarity with Delta Lake, Iceberg, or Hudi.
• Experience with CI/CD pipelines for data workflows.
• Understanding of messaging/streaming systems (Kafka, Kinesis).
• Knowledge of DevOps and container tools (Docker).
  
  

• Strong analytical and problem-solving capabilities.
• Ability to work independently and as part of a collaborative team.
• Good communication and documentation skills.
• Ownership mindset and willingness to learn and adapt.
  
  

This job was posted by Shashank Patil from Oneture Technologies.