Data Engineer

We are seeking a highly motivated and skilled Data Engineer to join our dynamic data team. The ideal candidate will have a strong foundation in data structures and algorithms (DSA), be proficient in Python and SQL, and have hands-on experience with big data technologies like Spark and PySpark. You will be responsible for designing, building, and maintaining scalable data pipelines and systems to support analytics, machine learning, and operational reporting.

We are looking for an Immediate Joiner.

• Design, develop, and optimize data pipelines and ETL workflows.
• Exposure to AWS cloud platform
• Work with large-scale datasets using Spark/PySpark.
• Write clean, efficient, and modular Python code using OOPs principles.
• Implement exception handling, logging, and file handling for robust data processing.
• Develop and optimize complex SQL queries for data extraction and transformation.
• Utilize Pandas and NumPy for data manipulation and analysis.
• Work with Redis for caching and fast data retrieval.
• Use multiprocessing/multithreading techniques to build efficient data processes.
• Handle data in various formats (JSON, CSV, etc.) and work with Python libraries like datetime, os, sys, and traceback.
• Use Git for version control and collaboration, and execute Linux commands for system-level tasks.
• Collaborate closely with data scientists, analysts, and software engineers to deliver data solutions.

• Strong foundation in Data Structures and Algorithms (DSA).
• Proficiency in Python with an understanding of Object-Oriented Programming (OOP).
• Strong hands-on experience with SQL, Pandas, and NumPy.
• Experience with Apache Spark and PySpark for big data processing.
• Good understanding of exception handling, logging, and file operations in Python.
• Familiarity with multiprocessing and multithreading for building efficient applications.
• Proficient in handling JSON and using modules like os, sys, datetime, and traceback.
• Experience with Redis for fast data operations and caching.
• Comfortable using Git and executing Linux shell commands.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.

• Experience with orchestration tools like Airflow or Prefect.
• Familiarity with containerization tools like Docker.
• Understanding of CI/CD pipelines.