Senior Data Engineer

Category: Technology

Location:

Shuru is a self-managed technology team specializing in accelerating visions through product, technology, and AI leadership. With a focus on bespoke execution, we deliver impactful solutions that are scalable and designed for success. At Shuru, we deliver mobile solutions that meet and exceed customer expectations. Our collaborative and fast-paced environment encourages creativity and innovation.

Our Data Platform team is hiring a Senior Data Engineer to build and maintain scalable pipelines, shape our data architecture on Azure Databricks, and mentor other engineers.

Responsibilities

• Work closely with source system teams and reporting teams to gather, analyze, and translate data requirements into scalable data pipelines.
• Design, develop, and maintain robust ETL/ELT pipelines using PySpark, SQL, and Delta Lake on Azure Databricks.
• Ingest and integrate data from multiple sources including MariaDB, Azure Event Hubs, APIs, and flat file systems.
• Build and optimize data models across bronze, silver, and gold layers using Delta tables.
• Ensure data reliability, accuracy, and freshness through monitoring, testing, and validation.
• Apply and enforce data governance, security, and compliance policies throughout the pipeline lifecycle.
• Collaborate with analysts, data scientists, and engineers to enable downstream data usage.
• Promote engineering excellence through code reviews, documentation, CI/CD, and automation.
• Mentor junior engineers and provide guidance on design and best practice
• Explore emerging technologies such as Structured Streaming, and support their adoption where beneficial.
  
  

Requirements

Requirements:

• 5+ years of experience in data engineering or similar roles.
• At least 1 year in a senior or technical lead capacity.
• Proven experience with Azure Databricks, Delta Lake, and Spark-based data pipelines.
• Experience integrating data from multiple sources such as MariaDB, Event Hubs, APIs, or file systems.
• Exposure to version control (e.g., Git), CI/CD pipelines, and Agile methodologies.
• Familiarity with Structured Streaming and real-time data processing is a plus.
• Experience in fintech, broking, or financial services domains is an advantage
  
  

Benefits

Benefits:

• Competitive salary and benefits package.
• Opportunity to work with a team of experienced product and tech leaders.
• A flexible work environment with remote working options.
• Continuous learning and development opportunities.
• Chance to make a significant impact on diverse and innovative projects
  
  

Details