Free cookie consent management tool by TermsFeed Lead Data Engineer - Big Data & PySpark | Antal Tech Jobs
Back to Jobs
2 Days ago

Lead Data Engineer - Big Data & PySpark

decor
Bangalore, Karnataka, India
Information Technology
Full-Time
UST

Overview

Role Description

Key Responsibilities:

  • Big Data Architecture: Design, develop, and maintain scalable and distributed data architectures capable of processing large volumes of data.
  • Data Storage Solutions: Implement and optimize data storage solutions using technologies such as Hadoop, Spark, and PySpark.
  • PySpark Development: Develop and implement efficient ETL processes using PySpark to extract, transform, and load large datasets.
  • Performance Optimization: Optimize PySpark applications for better performance, scalability, and resource management.

Qualifications

  • Proven experience as a Big Data Engineer with a strong focus on PySpark.
  • Deep understanding of Big Data processing frameworks and technologies.
  • Strong proficiency in PySpark for developing and optimizing ETL processes and data transformations.
  • Experience with distributed computing and parallel processing.
  • Ability to collaborate in a fast-paced, innovative environment.

Skills

Pyspark,Big Data, Python,

Share job
Similar Jobs
View All
14 Hours ago
Microsoft Dynamics 365 F&O Functional Consultant
Information Technology
  • 3 - 7 Yrs
  • Pune
Job Summary: The Associate, IT ERP Specialist is responsible for providing support to internal and external users to use CECO’s ERP effectively to fulfill business objectives. The Associate, IT ERP Specialist will assist other IT ERP Specialists with...
decor
1 Day ago
DevOps Engineer
Information Technology
  • 4 - 7 Yrs
  • Chennai
Role Profile We are looking for a DevOps Engineer, this role combines the management application systems, deployment processes to ensure accurate and efficient releases of new features and the maintenance of uptime, performance, and reliability. ...
decor
1 Day ago
Full Stack Developer
Information Technology
  • 260000 - 350000 INR - Yearly
  • Bangalore, Karnataka, India
About the job:As a Full Stack Developer at Serpwize Technologies LLP, you will have the opportunity to work with a cutting-edge tech stack that includes PHP, MySQL, HTML, CSS, JavaScript, Python, MongoDB, AngularJS, Node.js, React, Bubble.io, WordPre...
decor
1 Day ago
Data Scientist - L2 - BLR
Information Technology
  • Bangalore, Karnataka, India
About CoffeeBeans ConsultingCoffeeBeans is a tech-driven software consulting company that helps businesses solve complex problems using modern data, AI, and engineering solutions. We blend deep technical expertise with a product mindset to build scal...
decor
1 Day ago
Full Stack Developer (AI Focus) Internship in Jammu, Himachal Pradesh, Haryana, Sahibzada Ajit Singh Nagar, Chandigarh, Punjab, Uttarakhand, Mohali, Chandigarh, Mohali
Information Technology
  • Bangalore, Karnataka, India
We are looking for a passionate and driven full stack development intern who has a strong interest in AI tools and modern development practices. This internship is ideal for candidates who love building projects, experimenting with AI-powered tools, ...
decor
1 Day ago
Interesting Job Opportunity: PHP Developer - MVC Frameworks
Information Technology
  • Bangalore, Karnataka, India
Mumbai (Andheri East) | 5 Days Work From OfficeWe're looking for an experienced Senior PHP Developer to build scalable, high-performance web applications.Roles And Responsibilities Create clean, robust, well performing, DRY code in PHP/ JavaScript, M...
decor
1 Day ago
Orangebits - Data Engineer - Google Cloud Platform
Information Technology
  • Bangalore, Karnataka, India
DescriptionExperience : 5 to 10 YearsLocation : HyderabadWork Mode : Hybrid (3 days WFO in a week)Key Responsibilities Design, develop, and maintain data pipelines and data processing workflows on GCP. Implement scalable solutions using BigQuery and ...
decor
1 Day ago
Software Engineer
Information Technology
  • Bangalore, Karnataka, India
Job Title: Software EngineerExperience: 2-4 yearsLocation: BangaloreDepartment: Product & TechnologyEmployment Type: Full Time, PermanentAbout Digitap.ai:DIGITAP.AI is an Enterprise SaaS company providing high-tech advanced AI/ML, Alternate Data Solu...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media