Azure Senior Data Engineer / 5 yrs - 8 years experience / Pune / Immediate Opening

Overview

Key Responsibilities:

• Extract and process data from multiple databases, APIs, and cloud platforms by establishing strong source connections.
• Design, develop, and maintain ETL pipelines for efficient data transformation and ingestion.
• Write and optimize complex SQL queries for data extraction, validation, and reporting.
• Utilize PySpark and Big Data technologies (Hadoop, Spark, Databricks, etc.) to process and analyze large datasets.
• Perform data analysis using Python (Pandas, NumPy, etc.) to uncover trends, patterns, and insights.
• Develop and maintain interactive dashboards and reports in Power BI (good to have).
• Work with cross-functional teams to understand business requirements and translate them into data-driven solutions.
• Experience in Data warehouse schema creation and optimization
• Ensure data accuracy, security, and compliance with company policies.
• Present analytical findings and recommendations to stakeholders in a clear and actionable manner. * Must-Have Skills Data Engineering & Orchestration: • Strong hands-on experience with data orchestration tools, especially Apache Airflow Programming & Scripting: • Python expert with real-world production experience • Strong proficiency in SQL and Scala Streaming & Messaging Systems: • Hands-on experience with Kafka and/or Kinesis • Real-time data streaming implementation in live projects Big Data & Analytics Platforms: • Apache Spark and/or Databricks • Experience with Microsoft Azure Fabric Data Architecture & Modeling: • Strong expertise in data modeling, schema design, and performance optimization Automation & Product Development: • Experience in automation, including automating ML/data pipelines • Proven ability to build data products or tools from scratch Cloud Platform: • Strong experience in Microsoft Azure Good-to-Have Skills • Exposure to AI/ML concepts • Experience working on AI-based products • Hands-on experience in taking models from development to production (MLOps exposure)

Key Responsibilities
• Lead the design and development of scalable, real-time data pipelines
• Architect and implement end-to-end data platforms on Azure
• Build and automate robust data orchestration workflows
• Collaborate with AI/ML teams to productionize data and model pipelines
• Mentor junior data engineers and provide technical guidance
• Ensure data quality, reliability, and performance at scale