Azure Data Engineer (Senior)

Core Skills

JD for Data Engineer (Databricks + PySpark + SQL)

• Over all 6 to 8 years of solid experience in data projects.
• Excellent Design, develop, and maintain robust ETL/ELT pipelines for data ingestion, transformation, and storage.
• Proficient in SQL and must worked on complex joins, Subqueries, functions, procedure
• Able to perform SQL tunning and query optimization without support.
• Design, develop, and maintain ETL pipelines using Databricks, PySpark to extract, transform, and load data from various sources.
• Must have good working experience on Delta tables, deduplication, merging with terabyte of data set
• Optimize and fine-tune existing ETL workflows for performance and scalability.
• Excellent knowledge in dimensional modelling and Data Warehouse
• Must have experience on working with large data set
• Experience working with batch and real-time data processing (Good to have).
• Implemented data validation, quality checks, and ensure adherence to security and compliance standards.
• Ability to develop reliable, secure, compliant data processing systems.
• Work closely with cross-functional teams to support data analytics, reporting, and business intelligence initiatives.
• One should be self-driven and work independently without support.