Python Bigdata Developer - Chennai/ Bangalore - Bellandur

Job Summary

Job Title: Data Engineer Location: TechM Blr ITPL Years of Experience: 7 10 Years Job Summary: We are seeking a highly skilled Data Engineer with a strong background in PySpark to join our dynamic team. The ideal candidate will have 7 10 years of experience in data engineering, with a proven track record of designing, building, and maintaining scalable data pipelines. You will play a crucial role in transforming raw data into actionable insights, enabling data driven decision making across the organization. Skills Python, Pyspark, Hadoop, HiveQL, Devops Responsibilities: Design, develop, and maintain robust data pipelines using PySpark to process large datasets efficiently. Collaborate with data scientists and analysts to understand data requirements and deliver high quality data solutions. Implement data integration processes from various sources, ensuring data quality and consistency. Optimize existing data workflows and troubleshoot performance issues to enhance data processing efficiency. Work with cloud based data platforms and tools to manage and store data securely. Document data engineering processes and maintain clear communication with stakeholders regarding project status and challenges. Stay updated with the latest industry trends and technologies in data engineering and big data. Mandatory Skills: Strong expertise in PySpark for data processing and transformation. Proficient in SQL and experience with relational databases. Experience with data warehousing solutions and ETL processes. Familiarity with cloud platforms such as AWS, Azure, or Google Cloud. Solid understanding of data modeling and data architecture principles. Ability to work in a fast paced environment and manage multiple priorities effectively. Preferred Skills: Experience with Apache Kafka or similar streaming technologies. Knowledge of machine learning concepts and frameworks. Familiarity with containerization technologies like Docker. Experience with version control systems such as Git. Strong analytical and problem solving skills. Qualifications: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. 7 10 years of experience in data engineering or a related role. Proven experience in building and optimizing data pipelines using PySpark. Excellent communication skills and the ability to work collaboratively in a team environment. If you are passionate about data engineering and possess the required skills, we invite you to apply and join our innovative team at TechM Blr ITPL.