Information Technology
Full-Time
Cynosure Corporate Solutions
Overview
Key responsibilities:
Skills & Experience:
- Working with clients to understand their data.
- Basedontheunderstanding you will be building the data structures and pipelines.
- You will be working on the application from end to end collaborating with UI and other development teams.
- You will be responsible for building the data pipelines to migrate and load the data into the HDFSeither on-prem or in the cloud.
- Developing Data ingestion/process/integration pipelines effectively.
- Creating Hive data structures,metadata and loading the data into data lakes / Big Data warehouse environments.
- Optimized(Performance tuning) many data pipelines effectively to minimize cost.
- Codeversioning control and git repository is up to date.
- You will be responsible for building and maintaining CI/CD of the data pipelines.
- You will be managing the unit testing of all data pipelines
Skills & Experience:
- Bachelor’s degree in computer science or related field.
- Minimum of 5+years working experience with Spark, Hadoop eco systems.
- Minimum of 4+years working experience on designing data streaming pipelines.
- Should be an expert in either Python/Scala/Java.
- Should have experience in Data Ingestion and Integration into data lake using hadoop ecosystem tools such as Sqoop, Spark, SQL, Hive, Airflow, etc..
- Should have experience optimizing (Performance tuning) data pipelines.
- Minimum experience of 3+ years on NoSQL and Spark Streaming.
- Knowledge of Kubernetes and Docker is a plus.
- Should have experience with Cloud services either Azure/AWS.
- Should have experience with on-prem distribution such as Cloudera/HortonWorks/MapR.
- Basic understanding of CI/CD pipelines.
- Basic knowledge of Linux environment and commands.
Similar Jobs
View All
5 Hours ago
Software Engineer - C#/Javascript
Information Technology
- 1 - 5 Yrs
- Anywhere in India/Multiple Locations
Job Description :
We are searching for a Software Engineer who is experienced in C#, .NET technologies, and web technologies (Angular, Aurelia). Join our collaborative team, where you'll work together on exciting projects, designing, building, and...
5 Hours ago
Node.js Developer
Information Technology
- 2 - 6 Yrs
- Anywhere in India/Multiple Locations
Job Description:
We are looking for a skilled Node.js Developer to join our backend engineering team. The ideal candidate will be responsible for building scalable server-side applications, working on API development, and integrating with database...
6 Hours ago
Frontend Developer
Information Technology
- 2 - 7 Yrs
- Anywhere in India/Multiple Locations
Experience Level: 2+ Years
Employment Type: Full-Time
About the Role: We are looking for a talented Frontend Developer with expertise in React.js to build interactive and responsive web interfaces. You will work closely with backend developers,...
1 Day ago
Business Analyst (PDM) (H/F) - AIRBUS INDIA PRIVATE LIMITED
Information Technology
Job Description: Role: Business Analyst (PDM) Description: The Business Analyst - PDM is part of IIDPPT siglum located in India. He/she is responsible for the PDM and PDM related legacy products here in India. Success of this position will requir...
1 Day ago
Automation Test Lead - Senior Associate - Operate
Information Technology
At PwC, our people in software and product innovation focus on developing cutting-edge software solutions and driving product innovation to meet the evolving needs of clients. These individuals combine technical experience with creative thinking to ...
1 Day ago
Canvas App, Power Apps, Power automation / Senior Software Engineer
Information Technology
Job DescriptionSome careers shine brighter than others.If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new ...
1 Day ago
CCaaS Business Analyst-Senior Associate
Information Technology
A career within Data and Analytics services will provide you with the opportunity to help organisations uncover enterprise insights and drive business results using smarter data analytics. We focus on a collection of organisational technology capabi...
1 Day ago
Sustainability- Technology & Innovation - Full Stack Developer (Gen AI & Cloud) - Senior Associate
Information Technology
At PwC, our people in software and product innovation focus on developing cutting-edge software solutions and driving product innovation to meet the evolving needs of clients. These individuals combine technical experience with creative thinking to ...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in