Lead Data Engineer

Job Title: Lead Data Engineer

Location: Bengalore

Years Of Experience : 10 - 14 Years of experience

Education : BE /BTech / ME / MTech / MCA

About the Role

We are seeking an experienced and motivated Lead Data Engineer to join our team. In this role, you will be instrumental in shaping and enhancing our data infrastructure, driving innovative data solutions, and ensuring data quality and reliability across the organization.

You will collaborate closely with cross-functional teams—including data science, analytics, software engineering, and product management—to deliver robust data solutions that enable data-driven decision-making.

Alongside technical leadership, you will also play a key role in people leadership—mentoring, guiding, and growing a high-performing team of data engineers while fostering a collaborative and inclusive work culture.

Roles & Responsibilities

Technical Leadership

• Design, build, and maintain scalable, reliable, and performant data pipelines and architectures.
• Oversee architectural decisions for data-related projects, aligning them with business and scalability goals.
• Implement best practices for data integrity, security, and governance.
• Drive technical direction and be hands-on with complex problem-solving when necessary.
• Develop and maintain frameworks for data quality, monitoring, and observability.
• Analyze and optimize performance of data pipelines and queries for efficiency and speed.
• Implement strategies such as partitioning, indexing, and caching to optimize data performance.
• Contribute to the evolution of the data strategy and technology stack.
• Actively participate in code reviews, ensuring adherence to best practices.
• Monitor, troubleshoot, and resolve production data pipeline issues in a timely manner.

People Leadership

• Lead, mentor, and develop a team of data engineers, fostering a high-performance and collaborative culture.
• Conduct regular one-on-ones, set clear performance goals, and support career growth.
• Partner with HR and leadership to hire, onboard, and retain top talent.
• Promote a culture of diversity, inclusion, and continuous learning within the team.

Collaboration & Ownership

• Work with stakeholders to understand business requirements and translate them into technical solutions.
• Take ownership of projects from concept to delivery, ensuring quality and timely execution.
• Demonstrate a proactive approach to identifying challenges and delivering innovative solutions that enhance system robustness.

Technical Skills

• Cloud Expertise:
• Azure services (Data Factory, Synapse Analytics, Databricks, Azure SQL, Data Lake, Function Apps, DevOps) OR AWS services (Glue, Redshift, S3, Lambda, EMR, Kinesis, CloudWatch)
• Data Engineering: ETL processes, data modeling, data warehousing.
• Expertise in Tools: Azure Data Factory(expert level) , Snowflake / Azure Synapse / AWS Redshift.
• Programming: SQL (expert level), Python, Scala, or Java.
• Data Observability & Governance: Experience with frameworks/tools like Monte Carlo, Databand.
• Version Control: Git or similar.
• Additional Tools: Apache Airflow, dbt (for orchestration & transformation).