Pune, Maharashtra, India
Information Technology
Full-Time
Accolite
Overview
About The Role
We are looking for a Lead Data Engineer to architect and guide the design and implementation of a global data handling and synchronization platform. In addition to being hands-on, you will provide technical leadership to a small team of data engineers and advise on master data management (MDM) best practices, ensuring compliance with global data residency and privacy requirements (e.g., GDPR).
Requirements
8+ years of experience as a Data Engineer, including at least 2 years in a technical leadership or lead role.
Pyspark optimization experience.
Strong expertise in Microsoft Azure data stack (Azure SQL, Data Lake, Data Factory, PySpark) and distributed data architectures.
Proven experience with Master Data Management (MDM) and cross-region data synchronization.
Familiarity with data privacy, security, and compliance (GDPR, etc.).
Proficiency in Python, SQL, and ETL tools.
Strong leadership, problem-solving, and communication skills.
Preferred
Experience with MS-SQL, Cosmos DB, Databricks, and event-driven architectures.
Knowledge of CI/CD and infrastructure-as-code (Azure DevOps, ARM/Bicep, Terraform).
Key Responsibilities
Lead the design and implementation of data pipelines for global and regional data synchronization (Azure SQL, Data Lake, Data Factory, PySpark, etc.).
Define the data architecture and drive MDM strategies for consistent, high-quality data across regions.
Develop and enforce standards for secure handling of PII and non-PII data, ensuring GDPR and other compliance. Guide and mentor data engineers, reviewing code and solutions to ensure best practices.
Partner with software architects, DevOps, and business stakeholders to integrate data flows with application logic and deployment pipelines.
Oversee monitoring, alerting, and documentation for data processes within the existing frameworks.
Provide technical guidance on data partitioning, replication, schema evolution, and data governance.
We are looking for a Lead Data Engineer to architect and guide the design and implementation of a global data handling and synchronization platform. In addition to being hands-on, you will provide technical leadership to a small team of data engineers and advise on master data management (MDM) best practices, ensuring compliance with global data residency and privacy requirements (e.g., GDPR).
Requirements
8+ years of experience as a Data Engineer, including at least 2 years in a technical leadership or lead role.
Pyspark optimization experience.
Strong expertise in Microsoft Azure data stack (Azure SQL, Data Lake, Data Factory, PySpark) and distributed data architectures.
Proven experience with Master Data Management (MDM) and cross-region data synchronization.
Familiarity with data privacy, security, and compliance (GDPR, etc.).
Proficiency in Python, SQL, and ETL tools.
Strong leadership, problem-solving, and communication skills.
Preferred
Experience with MS-SQL, Cosmos DB, Databricks, and event-driven architectures.
Knowledge of CI/CD and infrastructure-as-code (Azure DevOps, ARM/Bicep, Terraform).
Key Responsibilities
Lead the design and implementation of data pipelines for global and regional data synchronization (Azure SQL, Data Lake, Data Factory, PySpark, etc.).
Define the data architecture and drive MDM strategies for consistent, high-quality data across regions.
Develop and enforce standards for secure handling of PII and non-PII data, ensuring GDPR and other compliance. Guide and mentor data engineers, reviewing code and solutions to ensure best practices.
Partner with software architects, DevOps, and business stakeholders to integrate data flows with application logic and deployment pipelines.
Oversee monitoring, alerting, and documentation for data processes within the existing frameworks.
Provide technical guidance on data partitioning, replication, schema evolution, and data governance.
Similar Jobs
View All
3 Hours ago
Python Developer
Information Technology
- 3 - 6 Yrs
- Bangalore
Job Title: Senior Backend Engineer
Location: Bangalore | In-Office | Full-time
Experience: 3–6 Years
Working Days: 5.5 Days (5 Days WFO + Alternate Saturdays WFH)
Job Description
We are seeking a Senior Backend Engineer to design and devel...
1 Day ago
React Developer
Information Technology
- Pune, Maharashtra, India
We are looking for an Integration Engineer with expertise in Kafka Connect and Azure Integration Services to manage operations and deliver enhancements for enterprise integration solutions. This role will be responsible for ensuring the reliability,...
1 Day ago
IT Specialist | voco™ Amritsar
Information Technology
- Pune, Maharashtra, India
Reliable hardware and seamless software are at the heart of every day-to-day IHG experience. To deliver a truly first-class service, we need an IT Supervisor capable of installing new systems, boosting security and setting exceptional standards. Whi...
1 Day ago
Java Fullstack developer With React
Information Technology
- Pune, Maharashtra, India
Wissen Technology is Hiring for Java Full stack developer with React About Wissen Technology: At Wissen Technology, we deliver niche, custom-built products that solve complex business challenges across industries worldwide. Founded in 2015, our cor...
1 Day ago
Lead, Data Analyst
Information Technology
- Pune, Maharashtra, India
Hiring Manager: Harshh MakkarRecruiter: Radhika KapoorWe are seeking a highly motivated and detail-oriented professional to lead the definition, management, and evolution of business metadata, including business terms, rules, KPI definitions, and da...
1 Day ago
Full Stack Developer
Information Technology
- Pune, Maharashtra, India
Job Title: Full Stack Developer (Angular & Node.js)Experience: 3+ YearsJob SummaryWe are looking for a skilled and motivated Full Stack Developer with 3+ years of professional experience in Angular and Node.js to join our development team. The ideal...
1 Day ago
QA Engineer
Information Technology
- Pune, Maharashtra, India
We are now looking for a QA Engineer to join our incredibly talented, passionate team. You must have worked on mobile apps within a fast-moving business and be able to cope with pace and change.Responsibilities Design and develop robust automation f...
1 Day ago
Web Scraping Developer
Information Technology
- Pune, Maharashtra, India
Python web scraping will be responsible for efficient web scraping/web crawling and parsing. The candidate should have demonstrated experience in web scraping and data extraction, along with the ability to communicate effectively and adhere to set d...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in