Senior PySpark Data Engineer

Overview

Vacancy with a leading End to End IT Services firm specializing in managing everything from Concept to Design to Development to Deployment and IT staffing solutions, Business is totally dependent on the modern and innovative technologies, and the Company provides robust IT talents that connect human resource with technology needs.The Company has been operational since 2022 and is Head Quartered in Mumbai.

Vacancy :

Job Location: Pune ( Hybrid)

• We are seeking a Senior PySpark Data Engineer with extensive experience in developing, optimizing, and maintaining data processing jobs using PySpark. The ideal candidate will possess a robust background in SQL and ETL processes, along with proficiency in cloud platforms such as AWS or Azure. This role will require excellent analytical skills and the ability to communicate effectively with both technical and non-technical stakeholders.

Key Responsibilities:

• Design, develop, and optimize PySpark jobs for enhanced performance and scalability.
• Collaborate with data architects and business analysts to understand data requirements and translate them into technical specifications.
• Redesign and maintain complex SQL queries and stored procedures to support data extraction and transformation processes.
• Utilize ETL tools, specifically Informatica PowerCenter, to build effective data pipelines.
• Troubleshoot and resolve data quality issues and performance bottlenecks.
• Mentor and provide technical guidance to a team of developers to enhance productivity and code quality.
• Stay updated with new technologies and practices to continually improve data processing capabilities.

Qualifications:

• Education: Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• Experience: 5-8 years of experience in data engineering, with a strong focus on PySpark and ETL processes.

Technical Skills:

Must-Have:

• Extensive experience with PySpark, focusing on job optimization techniques.
• Proficiency in SQL, with experience in SQL Server, MySQL, or other relational databases.
• Strong knowledge of ETL concepts and tools, particularly Informatica PowerCenter and IDMC.
• Excellent analytical and troubleshooting skills.
• Strong communication skills for effective collaboration.

Good to Have:

• Basic knowledge of Unix commands and Shell scripting.
• Experience in leading and mentoring development teams.
• Familiarity with Azure/Fabric.

Job Type: Full-time

Pay: Up to ₹1,100,639.31 per year

Benefits:

• Cell phone reimbursement
• Flexible schedule
• Health insurance
• Internet reimbursement
• Life insurance
• Paid sick time
• Paid time off
• Provident Fund

Schedule:

• Day shift

Supplemental Pay:

• Yearly bonus

Application Question(s):

• Do you have minimum 5 years of Experience in Python SQL ETL Informatica PowerCenter AWS/Azure & IDMC

Work Location: In person