AppSierra Solutions - Data Engineer - Big Data

Description

We are seeking highly skilled Data Engineers with a strong background in Scala, Databricks, and Kafka to join our dynamic team. In this role, you will be responsible for designing, building, and maintaining robust and scalable data pipelines that are essential for driving data-driven decision-making processes in our organization.

As a Data Engineer, you will work closely with data scientists, analysts, and other stakeholders to ensure the timely and accurate delivery of data to support a variety of business needs. We are looking for individuals who have a passion for data and a deep understanding of data architecture and engineering principles. Your expertise in Scala will be crucial for building high-performance data processing applications, while your experience with Databricks will enable you to leverage the power of cloud computing for big data analytics. Additionally, your familiarity with Kafka will help in creating real-time data streaming solutions that will enhance our data architecture. Join us in our mission to harness the power of data and drive innovation in our company. If you are a proactive problem solver with a collaborative mindset, we want to hear from you and explore how you can contribute to our success.

Responsibilities

• Design, build, and maintain efficient and reliable data pipelines using Scala, Databricks, and Kafka.
• Collaborate with data scientists and analysts to understand data requirements and provide solutions that meet their needs.
• Optimize ETL processes for performance and scalability, ensuring high data quality and integrity.
• Implement data streaming solutions to facilitate real-time data processing and analytics initiatives.
• Monitor and troubleshoot data pipelines, addressing performance issues and minimizing downtime.
• Participate in code reviews and ensure adherence to best practices in data engineering and software development.
• Stay up-to-date with emerging technologies and industry trends to continuously improve data engineering practices.
  
  

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field; Master's degree preferred.
• 5-8 years of experience in data engineering or a related field with a focus on big data technologies.
• Strong proficiency in Scala and experience with functional programming paradigms.
• Hands-on experience with Databricks and Spark for data processing and analytics.
• Proficient in Kafka for building real-time data streaming applications.
• Solid understanding of data modeling, ETL processes, and data warehousing concepts.
• Experience with cloud-based data platforms, preferably AWS, Azure, or Google Cloud.
  
  

(ref:hirist.tech)