Free cookie consent management tool by TermsFeed Data Engineer II - Spark/HDFS/Python | Antal Tech Jobs
Back to Jobs
1 Day ago

Data Engineer II - Spark/HDFS/Python

decor
Bangalore, Karnataka, India
Information Technology
Full-Time
NTEK Solutions

Overview

Role Overview

Prana Tree LLC is looking for a skilled and motivated Data Engineer II to join our growing data team. In this role, you will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support data-driven decision-making across the organization. The ideal candidate should have a strong foundation in data engineering principles, hands-on experience with modern data technologies, and a passion for solving complex data challenges.

Key Responsibilities

  • Work with Product Owners and stakeholders to identify opportunities for leveraging company data to drive business solutions.
  • Design and scale data platforms to meet business and technical requirements.
  • Mine and analyze data from company databases to improve product development, marketing strategies, and business operations.
  • Actively participate in team meetings and workshops with clients to align data solutions with business needs.
  • Ensure high-quality code for data engineering solutions to facilitate production implementation.
  • Create and maintain technical product backlogs for data projects, ensuring timely resolution.
  • Assess the effectiveness and accuracy of new data sources and data gathering techniques.
  • Lead data mining and collection processes, ensuring data quality and integrity.
  • Develop custom data models and algorithms to analyze large datasets.
  • Collaborate with cross-functional teams to implement models and monitor outcomes.
  • Develop tools and processes to monitor and analyze model performance and data accuracy.
  • Architect robust data platforms across multiple cloud technologies based on client requirements.
  • Design reusable and scalable data pipelines to enhance data integration and processing.
  • Work closely with Data Engineering (DE), Data Analytics (DA), ETL, QA, and Application teams to remove roadblocks and ensure smooth data operations.
  • Align data projects with organizational goals and business strategies.

Skills & Qualifications

  • 4-7 years of experience in data engineering, having worked on large-scale data projects.
  • Bachelors or Masters degree in Computer Science, Engineering, Data Science, or a related field.
  • Strong problem-solving skills with a focus on product development.
  • Domain expertise in Big Data, Data Platforms, and Distributed Systems.
  • Proficiency in Java, Scala, or Python (hands-on experience with Apache Spark is essential).
  • Experience with data ingestion frameworks such as Apache Storm, Flink, or Spark Streaming.
  • Experience with streaming technologies like Kafka, Kinesis, Oplogs, Binlogs, or Debezium.
  • Strong database skills with experience in HDFS, Delta Lake, Iceberg, or Lakehouse architectures.

(ref:hirist.tech)
Share job
Similar Jobs
View All
12 Hours ago
MERN Stack Developer - Remote
Information Technology
  • 2 - 4 Yrs
  • Anywhere in India/Multiple Locations
Job responsibility: Hire and mentor a team of high-calibre front-end and Backend engineers Design and develop new user-facing full-stack applications. Building reusable components and front-end, backend libraries for future use Optimizing...
decor
1 Day ago
ScaleUp Technologies - Software Engineer - Golang
Information Technology
  • Bangalore, Karnataka, India
Skills Experience in modern RESTful API development In-depth expertise in microservices, Docker and Kubernetes Strong knowledge in software architecture, data structures / design patterns Strong knowledge of Go programming language, paradigms, c...
decor
1 Day ago
Republic World - Junior DevOps Engineer - Google Cloud Platform
Information Technology
  • Bangalore, Karnataka, India
DevOps Engineer (2 - 5 Years Exp.) | NoidaLocation : Sector 158, Noida | On-site Description : Are you a seasoned DevOps Engineer with 2-5 years of experience, ready to take ownership of large-scale infrastructure and cloud deployments?We're looking...
decor
1 Day ago
ScaleUp Technologies - .Net Developer - C#/Javascript
Information Technology
  • Gurugram, Haryana, India
Job DescriptionWe are looking for a highly skilled and versatile .NET Developer to join our engineering team. The ideal candidate will have a strong foundation in .NET Core and C#, experience with modern frontend frameworks, and a passion for buildi...
decor
1 Day ago
Interesting Job Opportunity: Azure Architect
Information Technology
  • Bangalore, Karnataka, India
Job DescriptionThe Azure Architect will offer technical leadership in Microsoft Cloud, Azure, and Azure DevOps. include overseeing new implementations, cloud migrations, DevOps practices, governance, and platform strategy, as well as delivering cust...
decor
1 Day ago
FINVASIA - Linux System Administrator - Virtualization
Information Technology
  • Bangalore, Karnataka, India
Job SummaryWe are looking for a knowledgeable and proactive Linux System and Networking Administrator with strong experience in virtualization (especially Proxmox and VMware vCenter) and a deep understanding of enterprise networking.You will be resp...
decor
1 Day ago
Republic World - Junior DevOps Engineer - Cloud Environment
Information Technology
  • Bangalore, Karnataka, India
We're Hiring | Junior DevOps Engineer (2-5 Years Exp)Location : Noida | Immediate Joiners PreferredWe are looking for a Junior DevOps Engineer with 2-5 years of experience to join our team and support our deployment processes and infrastructure.This...
decor
1 Day ago
Technical Lead - PHP
Information Technology
  • Bangalore, Karnataka, India
Tech Lead PHP (Laravel / Phalcon + Full Stack)Location : Ahmedabad.Experience : 68 Years.Job DescriptionWere looking for a Tech Lead who will actively do R&D, code hands-on, mentor developers, and drive the technical direction of our product platfor...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media