Free cookie consent management tool by TermsFeed Kafka Data Engineer | Antal Tech Jobs
Back to Jobs
4 Weeks ago

Kafka Data Engineer

decor
Information Technology
Full-Time
Virtusa

Overview

Job Description Kafka Data Engineer

Data Engineer to build and manage data pipelines that support batch and streaming data solutions. The role requires expertise in creating seamless data flows across platforms like Data Lake/Lakehouse in Cloudera, Azure Databricks, Kafka for both batch and stream data pipelines etc.

Responsibilities

Strong experience in develop, test, and maintain data pipelines (batch & stream) using Cloudera, Spark, Kafka and Azure services like ADF, Cosmos DB, Databricks, NoSQL DB/ Mongo DB etc.

Strong programming skills in spark, python or scala & SQL.

Optimize data pipelines to improve speed, performance, and reliability, ensuring that data is available for data consumers as required.

Create ETL pipelines for downstream consumers by transform data as per business logic.

Work closely with Data Architects and Data Analysts to align data solutions with business needs and ensure the accuracy and accessibility of data.

Implement data validation checks and error handling processes to maintain high data quality and consistency across data pipelines.

Strong analytical and problem solving skills, with a focus on optimizing data flows and addressing impacts in the data pipeline. Qualifications

8+ years of IT experience with at least 5+ years in data engineering and cloud-based data platforms.

Strong experience with Cloudera/any Data Lake, Confluent/Apache Kafka, and Azure Data Services (ADF, Databricks, Cosmos DB).

Deep knowledge of NoSQL databases (Cosmos DB, MongoDB) and data modeling for performance and scalability.

Proven expertise in designing and implementing batch and streaming data pipelines using Databricks, Spark, or Kafka.

Experience in creating scalable, reliable, and high-performance data solutions with robust data governance policies.

Strong collaboration skills to work with stakeholders, mentor junior Data Engineers, and translate business needs into actionable solutions.

Bachelors or masters degree in computer science, IT, or a related field.
Share job
Similar Jobs
View All
8 Hours ago
Full Stack Developer (React + Python + AWS)
Information Technology
  • 5 - 8 Yrs
  • Pune
About the Client: They are a Global leader in delivering cutting-edge inflight entertainment and connectivity (IFEC) solutions. About the Role: Roles and Responsibilites Lead Frontend Development (70%): Design, build, and optimize inte...
decor
15 Hours ago
Senior Full Stack Software Engineer
Information Technology
  • 5 - 10 Yrs
  • Bangalore
Job Title: Senior Full Stack Software Engineer Location: Bengaluru Experience: 5+ Years Job Type: Full Time What our client offer Our client believes in rewarding our employees for their hard work. Our client offer competitive sala...
decor
15 Hours ago
Technical Delivery Engineer
Information Technology
  • 10 - 15 Yrs
  • Bangalore, Gurgaon / Gurugram
Location: Bangalore / Gurugram Mode: Hybrid (2–3 days in office per week) Experience Required: 10+ years Employment Type: Permanent Role Overview As a Technical Delivery Engineer, you will be a key member of our infrastructure delivery t...
decor
16 Hours ago
Lead Endur Specialist
Information Technology
  • 13 - 20 Yrs
  • Bangalore
What we offer: Our Client, believes in rewarding our employees for their hard work. We offer competitive salaries, company pensions and performance related benefits. Our people can also take advantage of our extensive flexible benefits package and...
decor
1 Day ago
UI Developer
Information Technology
  • 2 - 5 Yrs
  • Mumbai
Location: Mumbai, India Job Type: Full-Time Department: Design & Front-End Development Role Summary We’re on the hunt for a detail-oriented and creative UI Developer to bring intuitive, responsive, and visually striking interfaces to life...
decor
1 Day ago
Full Stack Developer
Information Technology
  • 2 - 5 Yrs
  • Mumbai
Location: Mumbai, India Employment Type: Full-time Department: Engineering / Technology Role Overview We’re looking for a proactive and skilled Full Stack Developer to join our dynamic team. The ideal candidate will be passionate about bu...
decor
1 Day ago
Senior .Net Developer - C#/ASP/MVC
Information Technology
  • Mumbai, Maharashtra, India
MultiverseTech is seeking a highly skilled and experienced Senior .NET Developer for a contractual role. We're looking for a proactive problem-solver with a strong background in developing robust, scalable, and high-performance web applications. If ...
decor
1 Day ago
Dark Matter Technologies - SQL Developer
Information Technology
  • Mumbai, Maharashtra, India
Key Responsibilities Design, develop, and optimize stored procedures, functions, views, and scripts for SQL Server. Implement and manage database indexing and partitioning strategies to enhance performance. Monitor and troubleshoot SQL Server per...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media