Bangalore, Karnataka, India
Information Technology
Contract
Awign
Overview
Role: Azure Data Engineer
Experience: 4 + to 8 Years
Location: Remote
Shift timings : 11am - 8pm (IST)
Mandatory Skills: Azure Cloud Technologies, Azure Data Factory, Azure Databricks (Advance Knowledge), PySpark, CI/CD Pipeline (Jenkins, GitLab CI/CD or Azure DevOps), Data Ingestion, SOL
Seeking a skilled Data Engineer with expertise in Azure cloud technologies, data pipelines, and big data processing. The ideal candidate will be responsible for designing, developing, and optimizing scalable data solutions.
Responsibilities
Experience: 4 + to 8 Years
Location: Remote
Shift timings : 11am - 8pm (IST)
Mandatory Skills: Azure Cloud Technologies, Azure Data Factory, Azure Databricks (Advance Knowledge), PySpark, CI/CD Pipeline (Jenkins, GitLab CI/CD or Azure DevOps), Data Ingestion, SOL
Seeking a skilled Data Engineer with expertise in Azure cloud technologies, data pipelines, and big data processing. The ideal candidate will be responsible for designing, developing, and optimizing scalable data solutions.
Responsibilities
- Azure Databricks and Azure Data Factory Expertise: Demonstrate proficiency in designing, implementing, and optimizing data workflows using Azure Databricks and Azure Data Factory. Provide expertise in configuring and managing data pipelines within the Azure cloud environment.
- PySpark Proficiency: Possess a strong command of PySpark for data processing and analysis. Develop and optimize PySpark code to ensure efficient and scalable data transformations.
- Big Data & CI/CD Experience: Ability to troubleshoot and optimize data processing tasks on large datasets. Design and implement automated CI/CD pipelines for data workflows. This involves using tools like Jenkins, GitLab CI/CD, or Azure DevOps to automate the building, testing, and deployment of data pipelines.
- Data Pipeline Development & Deployment: Design, implement, and maintain end-to-end data pipelines for various data sources and destinations. This includes unit tests for individual components, integration tests to ensure that different components work together correctly, and end-to-end tests to verify the entire pipeline's functionality. Familiarity with Github/Repo for deployment of code Ensure data quality, integrity, and reliability throughout the entire data pipeline.
- Extraction, Ingestion, and Consumption Frameworks: Develop frameworks for efficient data extraction, ingestion, and consumption. Implement best practices for data integration and ensure seamless data flow across the organization.
- Collaboration and Communication: Collaborate with cross-functional teams to understand data requirements and deliver scalable solutions. Communicate effectively with stakeholders to gather and clarify data-related requirements.
- Bachelor’s or master’s degree in Computer Science, Data Engineering, or a related field.
- 4+ years of relevant hands-on experience in data engineering with Azure cloud services and advanced Databricks.
- Strong analytical and problem-solving skills in handling large-scale data pipelines.
- Experience in big data processing and working with structured & unstructured datasets.
- Expertise in designing and implementing data pipelines for ETL workflows.
- Strong proficiency in writing optimized queries and working with relational databases.
- Experience in developing data transformation scripts and managing big data processing using PySpark..
Similar Jobs
View All
1 Day ago
Devops Cloud Engineer
Information Technology
- 3000000 - 3500000 INR - Monthly
- 7 - 14 Yrs
- Mumbai
Minimum Experience/Training Required:
• At least 7 years of relevant experience, with a strong track record in deploying solutions/applications on AWS cloud environment
• Proven ability to work across structured, semi-structured, and unstructured d...
1 Day ago
eGrove Systems - Senior Python Developer - Django/Flask
Information Technology
- Bangalore, Karnataka, India
eGrove Systems Pvt Ltd is looking for Senior Python Developer to join its team of experts.Skill : Senior Python Developer.Exp : 4+Yrs.NP : Immediate to 15days.Location : Chennai/Madurai.Skills Requirement Hands-on software development skills, deep ...
1 Day ago
Java Developer - Associate
Space Exploration & Research, Information Technology
- Hyderabad, Telangana, India
At PwC, our people in software and product innovation focus on developing cutting-edge software solutions and driving product innovation to meet the evolving needs of clients. These individuals combine technical experience with creative thinking to ...
1 Day ago
Lead Software Engineer – Full Stack Java
Space Exploration & Research, Information Technology
- Hyderabad, Telangana, India
Full Time Pune Posted 14 seconds ago Website Trading TechnologiesMulti-asset platform for capital marketsWe are currently seeking driven and talented Senior Software Engineers who thrive on pushing the limits of technology to produce state-of-the-...
1 Day ago
Senior Data Engineer
Space Exploration & Research, Information Technology
- Hyderabad, Telangana, India
Job Title: Senior Data EngineerLocation: CoimbatoreExperience: 5+ YearsJob Type: Full-TimeKey Responsibilities Design, develop, and maintain robust data pipelines using Airflow and AWS services. Implement and manage data warehousing using Databrick...
1 Day ago
Senior Data Analyst
Information Technology
- Bangalore, Karnataka, India
Company : Fives India Engineering & Projects Pvt. Ltd.Job Title : Data Analyst/Senior Data Analyst (BI developer)Job Location : Chennai, Tamil Nadu, IndiaJob Department : ITEducational Qualification : BE/B.Tech/MCA from a reputed Institute in Comput...
1 Day ago
ServiceNow System Architect
Space Exploration & Research, Information Technology
- Hyderabad, Telangana, India
The OpportunityJoin SoftwareOne as a ServiceNow Platform Architect and play a pivotal role in shaping our ServiceNow practice. Drive innovation in service and project delivery, ensuring our solutions meet our clients' needs. Collaborate with cross-f...
1 Day ago
Technical Lead, Firmware Development
Space Exploration & Research, Information Technology
- Hyderabad, Telangana, India
Landis+Gyr is a leading global provider of integrated energy management solutions. We measure and analyze energy utilization to generate empowering analytics for smart grid and infrastructure management, enabling utilities and consumers to reduce en...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in