Senior Data Engineer

We are looking for a Data Engineer SDE-3 who can take ownership of designing, developing, and maintaining scalable and reliable data pipelines. You will play a critical role in shaping the data infrastructure that powers business insights, product intelligence, and scalable learning platforms at PW.

Roles Open: Data Engineer SDE-2 and Data Engineer SDE-3

Location: Noida & Bangalore

Key Responsibilities:

• Design and implement scalable, efficient ETL/ELT pipelines to ingest, transform, and process data from multiple sources.
• Architect and maintain robust data lake and warehouse solutions, aligning with business and analytical needs.
• Own the development and optimization of distributed data processing systems using Spark, AWS EMR, or similar technologies.
• Collaborate with cross-functional teams (data science, analytics, product) to gather requirements and implement data-driven solutions.
• Ensure high levels of data quality, security, and availability across systems.
• Evaluate emerging technologies and tools for data processing and workflow orchestration.
• Build reusable components, libraries, and frameworks to enhance engineering efficiency and reliability.
• Drive performance tuning, cost optimization, and automation of data infrastructure.
• Mentor junior engineers, review code, and set standards for development practices.

Required Skills & Qualifications:

• 5+ years of professional experience in data engineering or backend systems with a focus on scalable systems.
• Strong hands-on experience with Python or Scala, and writing efficient, production-grade code.
• Deep understanding of data engineering concepts: data modeling, data warehousing, data lakes, streaming vs. batch processing, and metadata management.
• Solid experience with AWS (S3, Redshift, EMR, Glue, Lambda) or equivalent cloud platforms.
• Experience working with orchestration tools like Apache Airflow (preferred) or similar (Azkaban, Luigi).
• Proven expertise in working with big data tools such as Apache Spark, and managing Kubernetes clusters.
• Proficient in SQL and working with both relational (Postgres, Redshift) and NoSQL (MongoDB) databases.
• Ability to understand API-driven architecture and integrate with backend services as part of data pipelines.
• Strong problem-solving skills, with a proactive attitude towards ownership and continuous improvement.