Free cookie consent management tool by TermsFeed Data Engineer | Antal Tech Jobs
Back to Jobs
4 Weeks ago

Data Engineer

decor
Hyderabad, Telangana, India
Information Technology
Full-Time
IOWeb3 Technologies

Overview

Job Description

  • Lead the design, development, and implementation of high-performance, scalable, and resilient data pipelines for both batch and real-time (stream) data processing.
  • Utilize your strong expertise in Apache Spark with Python to build efficient data transformation and processing jobs.
  • Ensure data pipeline solutions are fault-tolerant and reliable, guaranteeing data quality and integrity throughout the data lifecycle.
  • Work extensively with cloud-native data services, specifically demonstrating a minimum of 3+ years of hands-on experience with AWS services such as S3, DMS, Redshift, Glue, Lambda, Kinesis, MSK, or equivalent services from Azure/GCP.
  • Develop and optimize complex SQL queries and work with various NoSQL technologies to manage and retrieve data efficiently.
  • Collaborate with cross-functional teams, including data scientists, analysts, and other engineers, to understand data requirements and deliver impactful data solutions.
  • Participate in data modeling, schema design, and data governance initiatives.
  • Monitor, troubleshoot, and optimize existing data pipelines and infrastructure for performance and cost efficiency.
  • Contribute to best practices for data engineering, ensuring maintainability, scalability, and security of our data platforms.
  • Leverage your experience across multiple domains to adapt and apply best practices to diverse data We're Looking For :
  • Minimum of 6 to 10 years of progressive experience building data solutions within Big Data environments.
  • A strong ability to build robust, resilient, scalable, fault-tolerant, and reliable data pipelines.
  • Mandatory hands-on experience with Apache Spark using Python for both batch and stream data processing.
  • Solid knowledge and practical experience in both batch and stream data processing methodologies.
  • Demonstrated exposure to working on data projects across multiple domains.
  • Strong hands-on capabilities with both SQL and NoSQL technologies.
  • Must have a minimum of 3+ years of hands-on experience with AWS services like S3, DMS, Redshift, Glue, Lambda, Kinesis, MSK, or similar data-focused services from Azure/GCP.
  • Excellent problem-solving, analytical, and debugging skills.
  • Ability to work independently and collaboratively in a remote team environment.
  • Strong communication and interpersonal skills.

(ref:hirist.tech)
Share job
Similar Jobs
View All
1 Day ago
Head of AI Cybersecurity
Information Technology
  • 8 - 13 Yrs
  • Chennai, Mumbai (All Areas)
Location: Mumbai & Chennai- Hybrid (1 week WFO in a month) | Experience: 8+ years Type: Full-time Seeking a visionary and hands-on Head of AI Cybersecurity to lead the end-to-end security of AI/ML systems. This role will drive strategy, implement...
decor
1 Day ago
Intern-AI Engineer
Finance & Banking
  • Pune, Maharashtra, India
Scopely is looking for an Intern - AI Engineer to join our Gen AI team in Bangalore!At Scopely, we care deeply about what we do and want to inspire play, every day - whether in our work environments alongside our talented colleagues, or through our ...
decor
1 Day ago
Software Developer - PHP/Laravel
Finance & Banking
  • Pune, Maharashtra, India
Software Developer (Laravel/PHP)Location : MumbaiWorking Mode : On-siteExperience : 1-5 YearsAbout The RoleKey Responsibilities : Collaborate with cross-functional teams to understand project requirements, define scope, and determine best practices...
decor
1 Day ago
GenAI Data Scientist - Manager
Finance & Banking
  • Pune, Maharashtra, India
At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals. In data...
decor
1 Day ago
Java Developer - C11
Finance & Banking
  • Pune, Maharashtra, India
The Applications Development Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technolo...
decor
1 Day ago
Trademo - Lead Data/AI Engineer
Finance & Banking
  • Pune, Maharashtra, India
About TrademoAt Trademo, we are transforming global trade and supply chains by leveraging cutting-edge AI technology to provide businesses with unparalleled visibility, compliance, and intelligence solutions.Our AI-driven platform simplifies the com...
decor
1 Day ago
Senior Web Developer- C11 - CHENNAI
Finance & Banking
  • Pune, Maharashtra, India
The Digital S/W Engineer Intmd Analyst is a developing professional role. Deals with most problems independently and has some latitude to solve complex problems. Integrates in-depth specialty area knowledge with a solid understanding of industry sta...
decor
1 Day ago
GenAI Data Scientist - Manager
Finance & Banking
  • Pune, Maharashtra, India
At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals. In data...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media