Lead Data Engineer

Lead Data Engineer

Position Summary

We are seeking a highly experienced Lead Data Engineer to drive the design, development, and evolution of scalable data platforms on Azure. In this role, you will lead the architecture and implementation of modern data pipelines, real-time data processing systems, and lakehouse solutions, enabling efficient ingestion, transformation, and storage of structured and unstructured data. You will work closely with data architects, business stakeholders, and engineering teams to build reliable, high-performance data solutions that support advanced analytics and business decision-making.

In addition to strong technical leadership, this role includes people management and delivery ownership for a Scrum team of data engineers. You will be responsible for guiding the team, driving execution, ensuring high-quality deliverables, and fostering best practices across data engineering, governance, and operational excellence. The ideal candidate combines deep expertise in big data technologies, Azure data ecosystem, and data modeling with strong leadership, communication, and stakeholder management skills to successfully deliver complex, enterprise-scale data initiatives.

Preferred Qualifications :

• Bachelor's degree in computer science, Engineering, or equivalent experience.

Must have :

• 12+ years of experience in enterprise software development. Min 8+ years of experience in data engineering, or data platform design.
• 5+ years of building data pipelines for structured and un-structured data.
• 2+ years of experience of azure data pipeline development.
• 3 + years of experience with Hadoop, Azure Databricks, Stream Analytics, Eventhub, Kafka.
• 3+ years of experience of relational and dimensional data modeling with SQL Server and data warehouse.
• 3+ years of experience with SQL Server, T-SQL, stored procedure development and database optimization.
• 3+ years of experience with MS BI stack (SSIS/SSRS/SSAS, Power BI).
• 2 + years of experience with Azure data platform (Azure data factory, Azure data lake, Azure SQL DB, Azure DW/Synapse).
• 2 + years of experience with Azure ML, Python, Scala
• 3+ years of experience with big data technologies.
• 2 + years of experience with Power BI, Azure Analysis Services
• Experience with working with business stakeholders, requirements & use case analysis.
• Strong communication and collaboration skills with creative problem-solving skills.

ESSENTIAL DUTIES AND RESPONSIBILITIES

• Participate, collaborate and lead team of data engineers
• Understand and review the business use cases for data pipelines for the Data Lake including ingestion, transformation and storing in the Lakehouse.
• Design and develop highly optimized and reliable enterprise level data pipelines to move data to data warehouse in real-time.
• Review and analyze existing business process, identify any gaps, and recommend improvements.
• Design and develop modern real-time solutions with Scala, Spark, Azure Databricks.
• Improve data governance and operational excellence.
• Develop data expertise and take ownership of data quality for the newly developed pipelines.
• Adhere to company coding standards and best practices.
• Conduct code reviews and create technical solution document.
• Work closely with engineering manager and leadership to align with technical execution of the project(s)
• Work closely with Data Architects to understand technical roadmap and able to influence if any gaps in the implementation.

Our Interview Practices

To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts. Our interview process is designed to assess your individual skills, experiences, and communication style. We value authenticity and want to ensure we’re getting to know you—not a digital assistant. To help maintain this integrity, we ask to remove virtual backgrounds and include in-person interviews in our hiring process. Please note that use of AI-generated responses or third-party support during interviews will be grounds for disqualification from the recruitment process.

Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.