Senior Data Engineer - AWS Databricks

Company Description

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visit www.blend360.com

Job Description

We are looking for an experienced Senior Data Engineer with a strong foundation in Python, SQL, and Spark, and hands-on expertise in AWS, Databricks. In this role, you will build and maintain scalable data pipelines and architecture to support analytics, data science, and business intelligence initiatives. You’ll work closely with cross-functional teams to drive data reliability, quality, and performance.

Responsibilities

• Design, develop, and optimize scalable data pipelines using Databricks in AWS such as Glue, S3, Lambda, EMR, Databricks notebooks, workflows and jobs.
• Building data lake in WS Databricks.
• Build and maintain robust ETL/ELT workflows using Python and SQL to handle structured and semi-structured data.
• Develop distributed data processing solutions using Apache Spark or PySpark.
• Partner with data scientists and analysts to provide high-quality, accessible, and well-structured data.
• Ensure data quality, governance, security, and compliance across pipelines and data stores.
• Monitor, troubleshoot, and improve the performance of data systems and pipelines.
• Participate in code reviews and help establish engineering best practices.
• Mentor junior data engineers and support their technical development.
  
  

Qualifications

Requirements

• Bachelor's or master's degree in computer science, Engineering, or a related field.
• 5+ years of hands-on experience in data engineering, with at least 2 years working with AWS Databricks.
• Strong programming skills in Python for data processing and automation.
• Advanced proficiency in SQL for querying and transforming large datasets.
• Deep experience with Apache Spark/PySpark in a distributed computing environment.
• Solid understanding of data modelling, warehousing, and performance optimization techniques.
• Proficiency with AWS services such as Glue, S3, Lambda and EMR.
• Experience with version control Git or Code commit
• Experience in any workflow orchestration like Airflow, AWS Step funtions is a plu