Free cookie consent management tool by TermsFeed Senior Data Engineer - Scala/Spark | Antal Tech Jobs
Back to Jobs
1 Day ago

Senior Data Engineer - Scala/Spark

decor
Hyderabad, Telangana, India
Space Exploration & Research, Information Technology
Full-Time
CognitivZen Technologies Private Limited

Overview

Job Summary

We are seeking a highly skilled Senior Data Engineer with a minimum of 7 years of experience to design, develop, and maintain robust data pipelines and real-time streaming applications. The ideal candidate will have strong expertise in Java, Kafka, AWS infrastructure (especially Lambda), Spark (Java or Scala), and experience working with API Gateways for modern data integration and delivery.

  • Design and implement scalable, high-performance data pipelines using Apache Kafka and AWS services.
  • Develop real-time streaming applications using Apache Spark (Java/Scala).
  • Build and maintain cloud-native solutions leveraging AWS Lambda, API Gateway, S3, Athena, CloudWatch, etc.
  • Collaborate with data scientists, analysts, and cross-functional teams to support data delivery and integration needs.
  • Design APIs and integrate with API gateways to expose and consume data services securely.
  • Monitor data pipeline performance and troubleshoot data quality and processing issues.
  • Optimize data processing and transformations for performance, scalability, and cost-efficiency.
  • Follow best practices for CI/CD, testing, versioning, and deployment of data components.

Required Skills

  • 7+ years of hands-on experience in Data Engineering roles.
  • Strong programming skills in Java (mandatory),Spring, Scala (preferred for Spark), or both.
  • In-depth experience with Apache Kafka for stream processing.

Strong Hands-on Expertise In AWS Services Including

  • Lambda
  • API Gateway
  • S3, CloudFormation, Athena, etc.
  • Solid experience with Apache Spark using either Java or Scala.
  • Experience working with API Gateways and RESTful APIs.
  • Strong understanding of data architecture, ETL processes, and data modeling.

Nice To Have

  • Experience with containerization (Docker, ECS, EKS).
  • Familiarity with CI/CD pipelines and tools like Jenkins, GitHub Actions.
  • Knowledge of data security and governance practices.

(ref:hirist.tech)
Share job
Similar Jobs
View All
1 Day ago
Devops Cloud Engineer
Information Technology
  • 3000000 - 3500000 INR - Monthly
  • 7 - 14 Yrs
  • Mumbai
Minimum Experience/Training Required: • At least 7 years of relevant experience, with a strong track record in deploying solutions/applications on AWS cloud environment • Proven ability to work across structured, semi-structured, and unstructured d...
decor
1 Day ago
eGrove Systems - Senior Python Developer - Django/Flask
Information Technology
  • Bangalore, Karnataka, India
eGrove Systems Pvt Ltd is looking for Senior Python Developer to join its team of experts.Skill : Senior Python Developer.Exp : 4+Yrs.NP : Immediate to 15days.Location : Chennai/Madurai.Skills Requirement Hands-on software development skills, deep ...
decor
1 Day ago
Java Developer - Associate
Space Exploration & Research, Information Technology
  • Hyderabad, Telangana, India
At PwC, our people in software and product innovation focus on developing cutting-edge software solutions and driving product innovation to meet the evolving needs of clients. These individuals combine technical experience with creative thinking to ...
decor
1 Day ago
Lead Software Engineer – Full Stack Java
Space Exploration & Research, Information Technology
  • Hyderabad, Telangana, India
Full Time Pune Posted 14 seconds ago Website Trading TechnologiesMulti-asset platform for capital marketsWe are currently seeking driven and talented Senior Software Engineers who thrive on pushing the limits of technology to produce state-of-the-...
decor
1 Day ago
Senior Data Engineer
Space Exploration & Research, Information Technology
  • Hyderabad, Telangana, India
Job Title: Senior Data EngineerLocation: CoimbatoreExperience: 5+ YearsJob Type: Full-TimeKey Responsibilities Design, develop, and maintain robust data pipelines using Airflow and AWS services. Implement and manage data warehousing using Databrick...
decor
1 Day ago
Senior Data Analyst
Information Technology
  • Bangalore, Karnataka, India
Company : Fives India Engineering & Projects Pvt. Ltd.Job Title : Data Analyst/Senior Data Analyst (BI developer)Job Location : Chennai, Tamil Nadu, IndiaJob Department : ITEducational Qualification : BE/B.Tech/MCA from a reputed Institute in Comput...
decor
1 Day ago
ServiceNow System Architect
Space Exploration & Research, Information Technology
  • Hyderabad, Telangana, India
The OpportunityJoin SoftwareOne as a ServiceNow Platform Architect and play a pivotal role in shaping our ServiceNow practice. Drive innovation in service and project delivery, ensuring our solutions meet our clients' needs. Collaborate with cross-f...
decor
1 Day ago
Technical Lead, Firmware Development
Space Exploration & Research, Information Technology
  • Hyderabad, Telangana, India
Landis+Gyr is a leading global provider of integrated energy management solutions. We measure and analyze energy utilization to generate empowering analytics for smart grid and infrastructure management, enabling utilities and consumers to reduce en...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media