Lead Data Engineer

Overview

Position: Lead Data Engineer

Experience: 8+ years

Qualification: Bachelor’s or master’s degree in computer science, Software Engineering, or a related field

Location: Gurgaon (Currently Work from Home till further notice)

We are seeking a Lead Data Engineer to guide and grow our data engineering practice. The ideal candidate brings deep technical expertise in SQL, Python, and PySpark, along with proven experience in large-scale data modeling, data migration, and data governance. You will lead a team of data engineers, mentor junior members, and play a key role in delivering scalable, secure, and high-performance data solutions across the organization.

Key Responsibilities

· Lead the design and implementation of scalable data pipelines using PySpark, SQL, and Databricks.

• Architect and implement enterprise-grade data models supporting analytics, machine learning, and business operations.
• Drive and oversee large-scale data migration projects ensuring minimal downtime and data integrity.
• Enforce and enhance data governance policies including data lineage, cataloging, quality, and access control.
• Collaborate with cross-functional teams to understand requirements and deliver data products at scale.
• Mentor and coach junior and mid-level engineers, providing technical leadership and performance feedback.
• Own the end-to-end delivery of data initiatives and ensure engineering best practices are followed.

Preferred Skills and Qualifications

• 8+ years of experience in data engineering with a minimum of 2 years in a technical leadership role.
• Expertise in SQL, Python, and PySpark with hands-on experience in building distributed data processing pipelines.
• In-depth knowledge of Apache Spark, Hive Metastore, and Spark-SQL integration.
• Proven track record of large-scale data modeling (dimensional, normalized, denormalized) and data migration across heterogeneous systems.
• Hands-on experience with Databricks and working knowledge of Delta Lake, Unity Catalog, and DBFS.
• Experience with data governance frameworks, tools, and practices—cataloging, classification, access controls, and compliance.
• Familiarity with Java for data integration and transformation tasks.
• Excellent communication and team leadership skills.

Nice to Have

• Experience with cloud platforms like Azure, AWS, or GCP.
• Exposure to streaming platforms such as Apache Kafka or Spark Streaming.
• Knowledge of data lineage tools (e.g., Apache Atlas, Collibra).
• Experience with CI/CD pipelines for data workflows using tools like GitHub Actions, Jenkins, or Azure DevOps.

Why Join Us?

• Lead a high-performing data engineering team on impactful, enterprise-scale projects.
• Shape the future of our data strategy and architecture.
• Flexible work environment with ample opportunities for leadership growth and innovation.

Job Type: Full-time

Pay: ₹3,000,000.00 - ₹5,000,000.00 per year

Benefits:

• Provident Fund
• Work from home

Schedule:

• Monday to Friday

Application Question(s):

• 8+ years in a data engineering

Education:

• Bachelor's (Required)

Work Location: Remote