Free cookie consent management tool by TermsFeed Estuate - Data Engineer/Integration Developer - Apache Spark | Antal Tech Jobs
Back to Jobs
1 Day ago

Estuate - Data Engineer/Integration Developer - Apache Spark

decor
Pune, Maharashtra, India
Information Technology
Full-Time
Estuate, Inc.

Overview

Key Responsibilities

  • Design, develop, and maintain scalable and reliable data pipelines and ETL processes.
  • Write, test, and deploy efficient Python code for data manipulation, scripting, and interacting with APIs.
  • Utilize Apache Airflow for scheduling, orchestrating, and monitoring complex data workflows.
  • Leverage Google Cloud Platform (GCP) services, including Google BigQuery (GBQ), for data storage, processing, and analytics.
  • Implement distributed computing solutions using Apache Spark to process and transform large volumes of data efficiently.
  • Develop secure data integrations by implementing authentication and authorization protocols like OAuth when interacting with various API endpoints.
  • Consume data from and integrate with diverse external and internal APIs.
  • Collaborate with data scientists, analysts, and other engineers to understand data needs and deliver high-quality data solutions.
  • Monitor pipeline performance, troubleshoot issues, and optimize data processes for efficiency and Skills and Experience :
  • Extensive experience in designing, building, and maintaining data pipelines, ETL processes, and data transformation workflows.
  • Proficiency in Python for scripting, data manipulation, and API interactions.
  • Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
  • Familiarity with Google Cloud Platform (GCP) services, including experience with Google BigQuery (GBQ).
  • In-depth knowledge and practical experience with Apache Spark for distributed data processing.
  • Experience with authentication/authorization protocols such as OAuth for secure API integrations.
  • Ability to consume and interact with various API endpoints for data Preferred Skills :
  • Experience working with and integrating data from Oracle EBS Financials.
  • Experience with the Ascend.io platform

(ref:hirist.tech)
Share job
Similar Jobs
View All
1 Day ago
Agira Technologies - Senior DevOps Engineer - Cloud Infrastructure
Information Technology
  • Pune, Maharashtra, India
Job Title : Senior DevOps Engineer / Principal DevOps Engineer.Location : Chennai.Experience : 8+ Years.Notice : Immediate to 15 days.Job DescriptionWe are seeking a Principal DevOps Engineer to drive the development, automation, and optimization of...
decor
1 Day ago
NetScore Technologies - .Net Developer
Information Technology
  • Pune, Maharashtra, India
NetScore Technologies, an award-winning Oracle NetSuite Partner, is currently looking for talented individuals to join our growing cloud ERP practice in Hyderabad.Headquartered in Vienna, Virginia with a strong focus on building AI powered cloud app...
decor
1 Day ago
WizCommerce - Technical Support Engineer - Ticketing System
Information Technology
  • Pune, Maharashtra, India
About The CompanyAt WizCommerce, we are revolutionising B2B commerce for manufacturers, wholesalers, and distributors across the USA, India and Canada. Our mission is to empower businesses with innovative technology solutions that drive revenue and ...
decor
1 Day ago
FictiveBox - Business Analyst
Information Technology
  • Pune, Maharashtra, India
We are seeking an experienced Business Analyst with a strong background in handling large-scale government projects. The role will begin with a mandatory 3-4 month on-site engagement at a designated government workshop, aimed at gaining an in-depth ...
decor
1 Day ago
Full Stack Developer - Python/React.js
Information Technology
  • Pune, Maharashtra, India
Leading role to handle team and project deliverables independently. Onboarding with project team as lead and ownership to each project life cycle implementation and deliverables. Collaborate with a cross-functional team to design, develop, and imp...
decor
1 Day ago
Jeavio - Senior Data Engineer - Python/ETL
Information Technology
  • Pune, Maharashtra, India
We are seeking an experienced Senior Data Engineer to join our team.The ideal candidate will have a strong background in data engineering and AWS infrastructure, with hands-on experience in building and maintaining data pipelines and the necessary i...
decor
1 Day ago
ELLKAY - System Administrator - IT Infrastructure
Information Technology
  • Pune, Maharashtra, India
Were looking for a skilled Mid-Level System Administrator to join our IT infrastructure team. The ideal candidate has hands-on experience managing both Windows and Linux systems, working with cloud platforms (Azure and AWS), and supporting productio...
decor
1 Day ago
Estuate - .Net Full Stack Developer - C#/AngularJS
Information Technology
  • Pune, Maharashtra, India
Key Responsibilities Design, develop, and support robust backend applications using .NET Core (6 or later) and C#. Build and maintain RESTful APIs and service architectures with a strong understanding of EF Core, LINQ, and MVC patterns. Develop d...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media