Data Engineer / 2-6 years experience

Overview

About Senzcraft:
Founded by IIM Bangalore and IEST Shibpur Alumni, Senzcraft is a hyper-automation company. Senzcraft vision is to Radically Simplify Today's Work. And Design Business Process For The Future. Using intelligent process automation technologies.

We have a suite of SaaS products and services, partnering with automation product companies.

Please visit our website - https://www.senzcraft.com for more details

Our AI Operations SaaS platform – https://MagicOps.ai

Senzcraft on linkedin -> https://www.linkedin.com/company/senzcraft

Senzcraft is awarded by Analytics India Magazine in it’s report “State of AI in India” as a “Niche AI startup”. Senzcraft is also recognized by NY based SSON as a top hyper-automation solutions provider.

About the Role (Data Engineer Engineer):

We are looking for a skilled Data Engineer with 2–6 years of experience to design, build, and optimize scalable data pipelines and data processing solutions. The ideal candidate should have strong experience in cloud platforms (GCP/AWS/Azure), Python, PySpark, SQL, Airflow, and big data technologies like Hadoop/Spark.
This role will involve working closely with data scientists, analysts, and business stakeholders to support end-to-end data engineering needs.

Key Responsibilities
• Design, develop, and maintain scalable data pipelines and ETL/ELT processes using Python, PySpark, and SQL.
• Build and optimize distributed data processing workflows using Spark and Hadoop ecosystems.
• Develop and orchestrate workflows using Apache Airflow or other scheduling tools.
• Work with cloud platforms (GCP, AWS, or Azure) to build and deploy data solutions.
• Implement data ingestion, transformation, and validation workflows from multiple structured and unstructured data sources.
• Ensure high standards of data quality, accuracy, and reliability across all pipelines.
• Optimize database queries, storage solutions, and processing engines for performance and cost efficiency.
• Collaborate with cross-functional teams to understand business requirements and translate them into scalable data engineering solutions.
• Implement best practices in data governance, security, monitoring, and documentation.
• Troubleshoot and resolve issues in production pipelines to maintain high availability.

Required Skills & Qualifications
Technical Skills
• Strong experience in Python for data processing and automation.
• Hands-on expertise in PySpark / Spark SQL.
• Advanced proficiency in SQL (query optimization, performance tuning).
• Experience with Airflow (DAG creation, scheduling, monitoring).
• Experience with at least one cloud platform:
o GCP (BigQuery, Dataflow, Dataproc, Cloud Storage)
o AWS (S3, EMR, Glue, Redshift, Lambda)
o Azure (ADF, Synapse, Databricks, ADLS)
• Knowledge of Hadoop ecosystem (HDFS, Hive, Spark, Oozie/YARN).
• Familiarity with CI/CD, Git, Docker (preferred).
Soft Skills
• Strong analytical and problem-solving skills.
• Good communication and stakeholder management skills.
• Ability to work in a fast-paced, agile environment.
• Strong ownership and accountability.
Education
• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

Preferred Qualifications
• Experience with Databricks or Snowflake.
• Exposure to Kafka or streaming data pipelines.
• Knowledge of data modeling and data warehousing concepts.
• Certifications in AWS/GCP/Azure (good to have).