Azure Data Engineer/Architect - Python/PySpark

About The Job

Work Mode : Hybrid

Mandatory Skills : Python, Pyspark, Airflow, Databricks, ETL, Data Pipelines, and Azure Databricks, Synapse, Data Factory and Data :

We are seeking a skilled and motivated Data Engineer with experience in Python, SQL, Azure, and cloud-based technologies to join our dynamic team. The ideal candidate will have a solid background in building and optimizing data pipelines, working with cloud platforms, and leveraging modern data engineering tools like Airflow, PySpark, and Azure Data Engineering. If you are passionate about data and looking for an opportunity to work on cutting-edge technologies, this role is for you!

Key Responsibilities

• Design, develop, and optimize data pipelines using Python, SQL, and PySpark.
• Collaborate with cross-functional teams to gather and understand data requirements.
• Implement and maintain data workflows using Apache Airflow and Azure Data Factory.
• Design and implement cloud-based data solutions in Microsoft Azure, ensuring scalability, performance, and security.
• Write efficient, scalable, and reusable Python code for data processing tasks.
• Work with large datasets, ensuring data is transformed, stored, and processed effectively.
• Utilize Azure services such as Azure Data Lake, Azure SQL Database, Azure Synapse, and Azure Databricks to build robust data solutions.
• Troubleshoot and resolve any issues with data pipelines and data flow across systems.
• Ensure high-quality standards for data management, integrity, and security.
• Stay up-to-date with the latest data engineering trends and technologies in the Azure cloud environment.
  
  

Key Skills & Qualifications

• Experience : 8-15 years of experience as a Data Engineer or related role.
• Programming : Proficiency in Python for data processing and automation tasks.
• SQL : Strong experience writing complex SQL queries for data manipulation and analysis.
• Cloud Technologies : Experience with Azure Cloud and services like Azure Data Lake, Azure SQL Database, Azure Synapse, Azure Databricks.
• Data Engineering Tools : Hands-on experience with Apache Airflow for orchestrating workflows and scheduling tasks.
• Big Data : Familiarity with PySpark for distributed data processing.
• Problem Solving : Strong analytical skills and the ability to troubleshoot data engineering challenges.
• Collaboration : Excellent communication skills and ability to work effectively in a team-oriented environment.
• Education : B.Tech in any branch or specialization
  
  

Skills : pipelines,data,data processing,sql,azure synapse,azure,azure data factory,airflow,etl,data data lake,azure data bricks,data warehouse,cloud,azure databricks,azure synapses,data engineering,data lake,skills

(ref:hirist.tech)