Panchkula, Haryana, India
Information Technology
Full-Time
DTechworks
Overview
Senior PySpark Data Engineer in Pune
Location : Pune/Hybrid
Experience : 5-8 years
Notice Period : Immediate to 15 days
Mandatory Skills : Python, SQL, ETL, Informatica PowerCenter, AWS/Azure
Good to Have : IDMC
Job Summary
We are seeking a Senior PySpark Data Engineer with extensive experience in developing, optimizing, and maintaining data processing jobs using PySpark. The ideal candidate will possess a robust background in SQL and ETL processes, along with proficiency in cloud platforms such as AWS or Azure. This role will require excellent analytical skills and the ability to communicate effectively with both technical and non-technical stakeholders.
Key Responsibilities
Education : Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
Experience : 5-8 years of experience in data engineering, with a strong focus on PySpark and ETL processes.
Technical Skills
Must-Have :
Location : Pune/Hybrid
Experience : 5-8 years
Notice Period : Immediate to 15 days
Mandatory Skills : Python, SQL, ETL, Informatica PowerCenter, AWS/Azure
Good to Have : IDMC
Job Summary
We are seeking a Senior PySpark Data Engineer with extensive experience in developing, optimizing, and maintaining data processing jobs using PySpark. The ideal candidate will possess a robust background in SQL and ETL processes, along with proficiency in cloud platforms such as AWS or Azure. This role will require excellent analytical skills and the ability to communicate effectively with both technical and non-technical stakeholders.
Key Responsibilities
- Design, develop, and optimize PySpark jobs for enhanced performance and scalability.
- Collaborate with data architects and business analysts to understand data requirements and translate them into technical specifications.
- Redesign and maintain complex SQL queries and stored procedures to support data extraction and transformation processes.
- Utilize ETL tools, specifically Informatica PowerCenter, to build effective data pipelines.
- Troubleshoot and resolve data quality issues and performance bottlenecks.
- Mentor and provide technical guidance to a team of developers to enhance productivity and code quality.
- Stay updated with new technologies and practices to continually improve data processing capabilities.
Education : Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
Experience : 5-8 years of experience in data engineering, with a strong focus on PySpark and ETL processes.
Technical Skills
Must-Have :
- Extensive experience with PySpark, focusing on job optimization techniques.
- Proficiency in SQL, with experience in SQL Server, MySQL, or other relational databases.
- Strong knowledge of ETL concepts and tools, particularly Informatica PowerCenter and IDMC.
- Excellent analytical and troubleshooting skills.
- Strong communication skills for effective collaboration.
- Basic knowledge of Unix commands and Shell scripting.
- Experience in leading and mentoring development teams.
- Familiarity with Azure/Fabric.
Similar Jobs
View All
17 Hours ago
Data Analyst (Odia Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Jharkhand, Andhra Pradesh, Odisha
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
17 Hours ago
Data Analyst (Kannada Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Karnataka, India
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
19 Hours ago
Technical Lead - Backend Development - Node.Js
Finance & Banking
- 50,00,000 - 55,00,000 INR - Annual
- 6 - 8 Yrs
- Bangalore
What youʼll be doing
We are much more than our job descriptions, but here is where you will begin:
● Collaborate with stakeholders, including product owners, project managers, and scrum masters, to define and clarify project requirements.
● Transl...
20 Hours ago
Engineering Manager
Finance & Banking
- 55,00,000 - 60,00,000 INR - Annual
- 8 - 12 Yrs
- Bangalore
What youʼll be doing
Weʼre much more than our job descriptions, but hereʼs where youʼll begin:
● Lead and deliver large-scale platform and product initiatives that impact millions of users.
● Collaborate with product, design, and business teams to...
1 Day ago
Technical Project Manager (WordPress)
Information Technology
- 6 - 10 Yrs
- Ahmedabad
Location: Ahmedabad / Remote
Experience: 6+ Years
About Us:
E2M Solutions is home to a growing remote team of WordPress experts building innovative digital solutions. We pride ourselves on delivering consistent value and excellence to our client...
1 Day ago
Senior WordPress Frontend Developer
Information Technology
- 5 - 10 Yrs
- Ahmedabad
At E2M Solutions, we're building a powerhouse remote team to deliver high-performing, user-centric WordPress solutions. If you live and breathe frontend development and are looking to work on cutting-edge WordPress projects with a passionate global t...
1 Day ago
Python Developer - C++/EDA
Information Technology
- Chennai, Tamil Nadu, India
Job DescriptionWe are seeking a highly skilled C++ Python Developer with a strong background in software development, scripting, and EDA tool integration. This role focuses on creating, enhancing, and maintaining tools used in silicon design and ver...
1 Day ago
Business Analyst
Information Technology
- Chennai, Tamil Nadu, India
Project Role : Business AnalystProject Role Description : Analyze an organization and design its processes and systems, assessing the business model and its integration with technology. Assess current state, identify customer requirements, and defin...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in