Free cookie consent management tool by TermsFeed Senior Azure Data Engineer - Python/Apache Spark | Antal Tech Jobs
Back to Jobs
2 Days ago

Senior Azure Data Engineer - Python/Apache Spark

decor
Chennai, Tamil Nadu, India
Information Technology
Full-Time
Multiverse Solutions Private Limited

Overview

Job Summary

We are seeking an experienced and hands-on Data Lead with deep expertise in Microsoft Azure Data Analytics ecosystem.

The ideal candidate will lead the design, development, and implementation of scalable data pipelines and analytics solutions using Azure Data Factory (ADF), Synapse Analytics, Microsoft Fabric, Apache Spark, and modern data modeling techniques.

A strong grasp of CDC mechanisms, performance tuning, and cloud-native architecture is Responsibilities :

  • Lead the architecture and implementation of scalable data integration and analytics solutions in Azure.
  • Design and build end-to-end data pipelines using ADF, Azure Synapse Analytics, Azure Data Lake, and Microsoft Fabric.
  • Implement and manage large-scale data processing using Apache Spark within Synapse or Fabric.
  • Develop and maintain data models using Star and Snowflake schema for optimal reporting and analytics performance.
  • Implement Change Data Capture (CDC) strategies to ensure near real-time or incremental data processing.
  • Collaborate with stakeholders to translate business requirements into technical data solutions.
  • Manage and mentor a team of data engineers and analysts.
  • Monitor, troubleshoot, and optimize performance of data workflows and queries.
  • Ensure best practices in data governance, security, lineage, and documentation.
  • Stay updated with the latest developments in the Azure data ecosystem and recommend Skills and Qualifications :
  • 8 to 10 years of overall experience in data engineering and analytics, with at least 3+ years in a data lead role.
  • Strong expertise in Azure Data Factory, Azure Synapse, and Microsoft Fabric.
  • Hands-on experience with Apache Spark for large-scale data processing.
  • Proficient in SQL, Python, or PySpark for data transformation and automation.
  • Solid experience with CDC patterns (e.g., using ADF, or SQL-based approaches).
  • In-depth understanding of data warehousing concepts and data modeling (Star, Snowflake).
  • Knowledge of Power BI integration with Synapse/Fabric is a plus.
  • Familiarity with DevOps for data pipelines, version control (Git), and CI/CD for data solutions.
  • Strong problem-solving skills and ability to lead architecture discussions and POCs.
  • Excellent communication and stakeholder management Qualifications :
  • Microsoft Certifications in Azure Data Engineering or Analytics.
  • Experience with Delta Lake, Databricks, or Snowflake (as source/target).
  • Knowledge of data privacy and compliance standards like GDPR, We Offer :
  • Opportunity to lead strategic data initiatives on the latest Azure stack.
  • A dynamic and collaborative work environment.
  • Access to continuous learning, certifications, and upskilling programs.
  • Competitive compensation and benefits package.

(ref:hirist.tech)
Share job
Similar Jobs
View All
1 Day ago
Java Full Stack Developer
Information Technology
  • Bangalore, Karnataka, India
Primary Skill - Java, Springboot, Angular, React.js Responsibilities: Lead and mentor a team of Java & Springboot Developers in the design, development, and maintenance of applications. Work with business stakeholders and technical teams to gather...
decor
1 Day ago
Python Developer, Analyst, Firm Risk Management
Information Technology
  • Bangalore, Karnataka, India
Morgan StanleyPython Developer - AnalystProfile DescriptionWe’re seeking someone to join our team as (Analyst - Python Developer) in (Credit Risk) division.Firm Risk ManagementIn the Firm Risk Management division, we advise businesses across the Fir...
decor
1 Day ago
Junior SQL Developer/Software Integration Specialist
Information Technology
  • Bangalore, Karnataka, India
Position : Junior SQL Developer / Software Integration SpecialistLocation : MumbaiExperience : Fresher or 1 Year of Relevant ExperienceJob Type : Full-timeDepartment : IT / DevelopmentKey skills : Sybase Power Builder/PB 6.5 /POWER BUILDERJob Overvi...
decor
1 Day ago
Agile Coach
Information Technology
  • Bangalore, Karnataka, India
Job PurposeResponsible to ensure a successful Agile adoption within the organization and thereby within the scrum teams and within the different organization disciplines. Lead the way to better deliver done-high-quality products, through continuous ...
decor
1 Day ago
Python Developer - Data Platform
Information Technology
  • Bangalore, Karnataka, India
Key Responsibilities Develop, maintain, and optimize robust data ingestion and ETL/ELT pipelines using Python on the Databricks platform. Work collaboratively with the architecture team to align implementation with long-term data strategies. Leve...
decor
1 Day ago
AWS Admin And AWS Architect
Information Technology
  • Bangalore, Karnataka, India
Primary skills:Technology->IOT Platform->AWS IOT A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to get to the heart of customer issues, diagnose problem areas, design innovative solutions and fa...
decor
1 Day ago
Machine Learning Engineer - Data Modeling
Information Technology
  • Bangalore, Karnataka, India
Role : Machine Learning and AI EngineerExp : 4+ YearsType : HybridLocation : Bangalore We are looking for a skilled Mid-Level AI Engineer with extensive experience in building full-stack AI solutions. The ideal candidate will have a strong foundat...
decor
1 Day ago
Interesting Job Opportunity: Target Integration - Principal Software Engineer - Node.js/Python
Information Technology
  • Chennai, Tamil Nadu, India
Key Responsibilities Drive the end-to-end architecture, design, and implementation of robust, scalable, and secure software systems. Define technical standards, frameworks, and best practices, ensuring delivery excellence. Collaborate with stakeh...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media