Free cookie consent management tool by TermsFeed Data Engineer - Apache Spark | Antal Tech Jobs
Back to Jobs
1 Day ago

Data Engineer - Apache Spark

decor
Bangalore, Karnataka, India
Information Technology
Full-Time
Estuate, Inc.

Overview

Key Responsibilities

  • Design, develop, and maintain scalable and reliable data pipelines and ETL processes.
  • Write, test, and deploy efficient Python code for data manipulation, scripting, and interacting with APIs.
  • Utilize Apache Airflow for scheduling, orchestrating, and monitoring complex data workflows.
  • Leverage Google Cloud Platform (GCP) services, including Google BigQuery (GBQ), for data storage, processing, and

analytics.

  • Implement distributed computing solutions using Apache Spark to process and transform large volumes of data efficiently.
  • Develop secure data integrations by implementing authentication and authorization protocols like OAuth when interacting

with various API endpoints.

  • Consume data from and integrate with diverse external and internal APIs.
  • Collaborate with data scientists, analysts, and other engineers to understand data needs and deliver high-quality data solutions.
  • Monitor pipeline performance, troubleshoot issues, and optimize data processes for efficiency and Skills and Experience :
  • Extensive experience in designing, building, and maintaining data pipelines, ETL processes, and data transformation workflows.
  • Proficiency in Python for scripting, data manipulation, and API interactions.
  • Hands-on experience with Apache Airflow for workflow orchestration and scheduling.
  • Familiarity with Google Cloud Platform (GCP) services, including experience with Google BigQuery (GBQ).
  • In-depth knowledge and practical experience with Apache Spark for distributed data processing.
  • Experience with authentication/authorization protocols such as OAuth for secure API integrations.
  • Ability to consume and interact with various API endpoints for data Preferred Skills :
  • Experience working with and integrating data from Oracle EBS Financials.
  • Experience with the Ascend.io platform

(ref:hirist.tech)
Share job
Similar Jobs
View All
1 Day ago
Sr. Business Analyst - Salesforce Job
Information Technology
  • Bangalore, Karnataka, India
We use cookies to offer you the best possible website experience. Your cookie preferences will be stored in your browser’s local storage. This includes cookies necessary for the website's operation. Additionally, you can freely decide and change any...
decor
1 Day ago
Predigle - AI Engineer - Machine Learning/Python
Information Technology
  • Bangalore, Karnataka, India
Job Description : AI Engineer GenAI & Applied MLLocation : Chennai / Remote-friendlyExperience Level : 13 YearsJob Type : Full-TimeAbout The RoleWe are seeking a highly motivated AI Engineer with a strong background in Generative AI, Large Langua...
decor
1 Day ago
Revalsys Technologies - Data Engineer - Big Data/Python
Information Technology
  • Bangalore, Karnataka, India
Key Responsibilities Design, develop, and optimize big data pipelines using Azure Databricks, PySpark, and Delta Lake. Collaborate with Azure services like Data Lake, Synapse, SQL Database, and Data Factory to implement robust data solutions. Dev...
decor
1 Day ago
MERN Stack Web Developer Internship in Jaipur
Information Technology
  • Bangalore, Karnataka, India
Selected Intern's Day-to-day Responsibilities Include Collaborate with our team to design and develop live web applications that make a real impact Dive into the MERN stack and turn design concepts into fully functional features Ensure that appli...
decor
1 Day ago
Atlas Systems - UI Developer - React.js Framework
Information Technology
  • Bangalore, Karnataka, India
About UsAtlas Systems Inc. is a Software Solutions company headquartered in East Brunswick, NJ.Incorporated in 2003, Atlas provides comprehensive range of solutions in the area of GRC, Technology, Procurement, Healthcare Provider and Oracle to custo...
decor
1 Day ago
Morningstar - L1 Security Analyst
Information Technology
  • Bangalore, Karnataka, India
Role : Security Analyst L1The Information Security Team is a central function governing corporate and product security globally. We have built a strong team of high performing security experts and are creating a new team within Information Security ...
decor
1 Day ago
iOS Developer Internship in Noida
Information Technology
  • Bangalore, Karnataka, India
Selected Intern's Day-to-day Responsibilities Include Assist in developing and maintaining iOS applications using Swift. Collaborate with design and backend teams to implement new features. Write clean, maintainable, and scalable code. Debug and...
decor
1 Day ago
BeBetta - Game Developer - Unity3D/C#
Information Technology
  • Bangalore, Karnataka, India
Job : Game Developer (Unity)Experience : 2 -4 : Bangalore (On-site)Work Days : 6 Days a WeekAbout BeBettaBeBetta is a gamified reward platform for Gamers and Entertainers. We're on a mission to merge play, performance, and purpose by rewarding engag...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media