Bangalore, Karnataka, India
Information Technology
Full-Time
IOWeb3 Technologies
Overview
Job Description
Responsibilities :
Responsibilities :
- Lead the design, development, and implementation of high-performance, scalable, and resilient data pipelines for both batch and real-time (stream) data processing.
- Utilize your strong expertise in Apache Spark with Python to build efficient data transformation and processing jobs.
- Ensure data pipeline solutions are fault-tolerant and reliable, guaranteeing data quality and integrity throughout the data lifecycle.
- Work extensively with cloud-native data services, specifically demonstrating a minimum of 3+ years of hands-on experience with AWS services such as S3, DMS, Redshift, Glue, Lambda, Kinesis, MSK, or equivalent services from Azure/GCP.
- Develop and optimize complex SQL queries and work with various NoSQL technologies to manage and retrieve data efficiently.
- Collaborate with cross-functional teams, including data scientists, analysts, and other engineers, to understand data requirements and deliver impactful data solutions.
- Participate in data modeling, schema design, and data governance initiatives.
- Monitor, troubleshoot, and optimize existing data pipelines and infrastructure for performance and cost efficiency.
- Contribute to best practices for data engineering, ensuring maintainability, scalability, and security of our data platforms.
- Leverage your experience across multiple domains to adapt and apply best practices to diverse data challenges.
- Minimum of 6 to 10 years of progressive experience building data solutions within Big Data environments.
- A strong ability to build robust, resilient, scalable, fault-tolerant, and reliable data pipelines.
- Mandatory hands-on experience with Apache Spark using Python for both batch and stream data processing.
- Solid knowledge and practical experience in both batch and stream data processing methodologies.
- Demonstrated exposure to working on data projects across multiple domains.
- Strong hands-on capabilities with both SQL and NoSQL technologies.
- Must have a minimum of 3+ years of hands-on experience with AWS services like S3, DMS, Redshift, Glue, Lambda, Kinesis, MSK, or similar data-focused services from Azure/GCP.
- Excellent problem-solving, analytical, and debugging skills.
- Ability to work independently and collaboratively in a remote team environment.
- Strong communication and interpersonal skills
Similar Jobs
View All
18 Hours ago
Principal Engineer - UI/UX
Information Technology
- 6 - 11 Yrs
- Mumbai (All Areas)
looking for a technically savvy and experienced Principal Engineer to take up front-end development efforts. You will design and develop elegant interfaces that exceed client expectations in terms of value and benefit. You will collaborate on scalabi...
18 Hours ago
Principal Engineer - Beckend
Information Technology
- 7 - 11 Yrs
- Mumbai (All Areas)
Principal Engineer - Backend
Mumbai, Maharashtra, India | Product Engineering | Full-time
We are looking for a technically savvy and experienced senior developer to lead development efforts. You will help the team grow in size and skills, optim...
18 Hours ago
Principal Engineer - DevOps/DBA
Information Technology
- 7 - 11 Yrs
- Mumbai (All Areas)
we are looking for a technically savvy and passionate Principal DevOps Engineer or Senior Database Administrator to cater to the development and operations efforts in product. You will choose and deploy tools and technologies to build and support a r...
20 Hours ago
C++ Developer | 5 - 8 Years
Information Technology
- 30 - 45 INR - Yearly
- 5 - 12 Yrs
- Pune
About the Client:
The company has been a global leader in delivering cutting-edge in-flight entertainment and connectivity (IFEC) solutions for over 40 years.
About the Role:
Title:
SDE 3 C++ - 5 to 8 Years
SDE 4 C++ - 8 to 13 Years
...
1 Day ago
Software Engineer II - Java, Python, AWS, Pyspark, DPL
Finance & Banking
- Bangalore, Karnataka, India
hackajob is collaborating with J.P. Morgan to connect them with exceptional tech professionals for this role.Organization DescriptionOur Consumer & Community Banking division serves our Chase customers through a range of financial services, includin...
1 Day ago
Lead Software Engineer- Devportal Node fullstack
Finance & Banking
- Bangalore, Karnataka, India
Reference 25000GPGResponsibilities In-depth knowledge of UI development with React JS, Java script, Type script including its core principles and workflows. Expertise on backend development with Node Js and framework nest JS Familiarity with version...
1 Day ago
QA Engineer (Manual & Automation) in Delhi, Faridabad, Gurgaon, Noida (Hybrid)
Finance & Banking
- Bangalore, Karnataka, India
We are looking for a talented 'QA engineer (automation + manual)' to join our team at AppQuarterz Technologies! As a 'QA engineer', you will play a crucial role in ensuring the quality and performance of our software products.Key Responsibilities D...
1 Day ago
Software Engineer III - Infrastructure
Finance & Banking
- Bangalore, Karnataka, India
hackajob is collaborating with J.P. Morgan to connect them with exceptional tech professionals for this role.Job DescriptionEmpowering projects with innovative technical solutions, our team excels in optimizing data and systems at scale while ensuri...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in