Pune, Maharashtra, India
Information Technology
Full-Time
CIGNEX
Overview
: Azure or AWS Data Engineer (Exp: 5 years)
Location: Bangalore - Hybrid, 3 days from office.
Technology: Databricks, Data Services, Python
No of Positions: 1
:
We are looking for a highly skilled Lead Data Engineer with expertise in Azure or AWS and Databricks to join our team. The ideal candidate will lead the design, development, and implementation of data engineering solutions, ensuring scalability, security, and efficiency in our data infrastructure. This role requires strong technical skills and experience in managing large-scale data processing pipelines.
Key Responsibilities
Location: Bangalore - Hybrid, 3 days from office.
Technology: Databricks, Data Services, Python
No of Positions: 1
:
We are looking for a highly skilled Lead Data Engineer with expertise in Azure or AWS and Databricks to join our team. The ideal candidate will lead the design, development, and implementation of data engineering solutions, ensuring scalability, security, and efficiency in our data infrastructure. This role requires strong technical skills and experience in managing large-scale data processing pipelines.
Key Responsibilities
- Lead the design and development of scalable and reliable data pipelines using Azure Data Services or AWS Data Services and Databricks.
- Architect, implement, and optimize ETL/ELT processes to process large volumes of structured and unstructured data.
- Develop and maintain data models, data lakes, and data warehouses to support analytics and business intelligence needs.
- Collaborate with data scientists, analysts, and business stakeholders to ensure data availability and integrity.
- Implement and enforce data governance, security, and compliance best practices.
- Optimize and monitor the performance of data processing frameworks (Spark, Databricks, etc.).
- Automate and orchestrate data workflows using tools such as Apache Airflow, Azure Data Factory, AWS Step Functions, or Glue.
- Guide and mentor junior data engineers in best practices and modern data engineering techniques.
- 5 years of experience in data engineering
- Strong expertise in Azure Data Services (Azure Data Lake, Azure Synapse, Azure Data Factory) or AWS Data Services (S3, Redshift, Glue, Lambda, Step Functions, EMR).
- Proficiency in Databricks and experience with Apache Spark for large-scale data processing.
- Strong programming skills in Python
- Experience working with SQL and NoSQL databases such as PostgreSQL, MySQL, DynamoDB, or CosmosDB.
- Solid understanding of data governance, security, and compliance (GDPR, HIPAA, etc.) is a plus.
- Experience with real-time streaming technologies such as Kafka, Kinesis, or Event Hubs is a plus.
- Excellent problem-solving skills and the ability to work in a fast-paced, agile environment.
- Experience with machine learning pipelines and MLOps.
- Familiarity with data visualization and BI tools like Power BI, Tableau, or Looker.
- Strong communication and leadership skills to drive best practices across the team.
Similar Jobs
View All
1 Day ago
Quality Engineering Lead (Test Lead)
Manufacturing & Industrial
- Hyderabad, Telangana, India
Project Role : Quality Engineering Lead (Test Lead)Project Role Description : Leads a team of quality engineers through multi-disciplinary team planning and ecosystem integration to accelerate delivery and drive quality across the application lifecy...
1 Day ago
Software Engineer III - Python and NoSQL
Manufacturing & Industrial
- Hyderabad, Telangana, India
FactSet creates flexible, open data and software solutions for over 200,000 investment professionals worldwide, providing instant access to financial data and analytics that investors use to make crucial decisions.At FactSet, our values are the foun...
1 Day ago
Data Engineer
Manufacturing & Industrial
- Hyderabad, Telangana, India
Join our TeamAbout this opportunity:We are seeking a driven and dynamic Integration Engineer to join our team at Ericsson. This role provides an exceptional opportunity for a motivated individual to contribute meaningfully to data-driven integration...
1 Day ago
IT&D Platform Manager- Controls, Risk and Compliance, Nutrition
Manufacturing & Industrial
- Hyderabad, Telangana, India
We are ReckittHome to the world's best loved and trusted hygiene, health, and nutrition brands. Our purpose defines why we exist: to protect, heal and nurture in the relentless pursuit of a cleaner, healthier world. We are a global team united by th...
1 Day ago
RPA Business Analyst
Manufacturing & Industrial
- Hyderabad, Telangana, India
Hello, FCM part of FTCG is one of the world’s largest travel management companies and a trusted partner for nationals and multinational companies. With a 24/7 reach in 97 countries, FCM’s flexible technology anticipates and solves client needs, supp...
1 Day ago
Business Analyst (S&D, Marketing – Retail Automation)
Manufacturing & Industrial
- Hyderabad, Telangana, India
Job DescriptionA Business analyst must design, plan, develop, and execute the business, financial, and operational systems. It is for supporting the fundamental functions and business processes of the organization. These business processes involve c...
1 Day ago
Software Engineer III - C++ Developer
Manufacturing & Industrial
- Hyderabad, Telangana, India
About NCR VOYIXNCR VOYIX Corporation (NYSE: VYX) is a leading global provider of digital commerce solutions for the retail, restaurant and banking industries. NCR VOYIX is headquartered in Atlanta, Georgia, with approximately 16,000 employees in 35 ...
1 Day ago
Senior Data Engineer
Manufacturing & Industrial
- Hyderabad, Telangana, India
Job FamilyData Science & Analysis (India)Travel RequiredNoneClearance RequiredNoneWhat You Will Do Design, develop, and maintain robust, scalable, and efficient data pipelines and ETL/ELT processes. Lead and execute data engineering projects from in...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in