Indore, Madhya Pradesh, India
Information Technology
Full-Time
MoreYeahs
Overview
We are seeking a highly skilled and experienced Lead Data Engineer (7+ years) to join our dynamic team. As a Lead Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure. You will be responsible for ensuring the efficient and reliable collection, storage, and transformation of large-scale data to support business intelligence, analytics, and data-driven decision-making.
Key Responsibilities
Data Architecture & Design :
Education: Bachelors or Masters degree in Computer Science, Information Systems, Engineering, or a related field.
Experience
Key Responsibilities
Data Architecture & Design :
- Lead the design and implementation of robust data architectures that support data warehousing (DWH), data integration, and analytics platforms.
- Develop and maintain ETL (Extract, Transform, Load) pipelines to ensure the efficient processing of large datasets.
- Design, develop, and optimize ETL processes using tools like Informatica Power Center, Intelligent Data Management Cloud (IDMC), or custom Python scripts.
- Implement data transformation and cleansing processes to ensure data quality and consistency across the enterprise.
- Build and maintain scalable data warehouse solutions using Snowflake, Databricks, Redshift, or similar technologies.
- Ensure efficient storage, retrieval, and processing of structured and semi-structured data.
- Utilize AWS Glue and PySpark for large-scale data processing and transformation.
- Implement and manage data pipelines using Apache Airflow for orchestration and scheduling.
- Leverage cloud platforms (AWS, Azure, GCP) for data storage, processing, and analytics.
- Establish and enforce data governance and security best practices.
- Ensure data integrity, accuracy, and availability across all data platforms.
- Implement monitoring and alerting systems to ensure data pipeline reliability.
- Work closely with data Stewards, analysts, and business stakeholders to understand data requirements and deliver solutions that meet business needs.
- Mentor and guide junior data engineers, fostering a culture of continuous learning and development within the team.
- Lead data-related projects from inception to delivery, ensuring alignment with business objectives and timelines.
- Design and manage relational databases (RDBMS) to support transactional and analytical workloads.
- Optimize SQL queries for performance and scalability across various database platforms.
Education: Bachelors or Masters degree in Computer Science, Information Systems, Engineering, or a related field.
Experience
- Minimum of 7+ years of experience in data engineering, ETL, and data warehouse development.
- Proven experience with ETL tools like Informatica Power Center or IDMC.
- Strong proficiency in Python and PySpark for data processing.
- Experience with cloud-based data platforms such as AWS Glue, Snowflake, Databricks, or Redshift.
- Hands-on experience with SQL and RDBMS platforms (e.g., Oracle, MySQL, PostgreSQL).
- Familiarity with data orchestration tools like Apache Airflow.
- Advanced knowledge of data warehousing concepts and best practices.
- Strong understanding of data modeling, schema design, and data governance.
- Proficiency in designing and implementing scalable ETL pipelines.
- Experience with cloud infrastructure (AWS, Azure, GCP) for data storage and processing.
- Excellent communication and collaboration skills.
- Ability to lead and mentor a team of engineers.
- Strong problem-solving and analytical thinking abilities.
- Ability to manage multiple projects and prioritize tasks effectively.
- Experience with machine learning workflows and data science tools.
- Certification in AWS, Snowflake, Databricks, or relevant data engineering technologies.
- Experience with Agile methodologies and DevOps practices.
Similar Jobs
View All
17 Hours ago
Data Analyst (Odia Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Jharkhand, Andhra Pradesh, Odisha
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
17 Hours ago
Data Analyst (Kannada Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Karnataka, India
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
19 Hours ago
Technical Lead - Backend Development - Node.Js
Finance & Banking
- 50,00,000 - 55,00,000 INR - Annual
- 6 - 8 Yrs
- Bangalore
What youʼll be doing
We are much more than our job descriptions, but here is where you will begin:
● Collaborate with stakeholders, including product owners, project managers, and scrum masters, to define and clarify project requirements.
● Transl...
20 Hours ago
Engineering Manager
Finance & Banking
- 55,00,000 - 60,00,000 INR - Annual
- 8 - 12 Yrs
- Bangalore
What youʼll be doing
Weʼre much more than our job descriptions, but hereʼs where youʼll begin:
● Lead and deliver large-scale platform and product initiatives that impact millions of users.
● Collaborate with product, design, and business teams to...
1 Day ago
Technical Project Manager (WordPress)
Information Technology
- 6 - 10 Yrs
- Ahmedabad
Location: Ahmedabad / Remote
Experience: 6+ Years
About Us:
E2M Solutions is home to a growing remote team of WordPress experts building innovative digital solutions. We pride ourselves on delivering consistent value and excellence to our client...
1 Day ago
Senior WordPress Frontend Developer
Information Technology
- 5 - 10 Yrs
- Ahmedabad
At E2M Solutions, we're building a powerhouse remote team to deliver high-performing, user-centric WordPress solutions. If you live and breathe frontend development and are looking to work on cutting-edge WordPress projects with a passionate global t...
1 Day ago
Python Developer - C++/EDA
Information Technology
- Chennai, Tamil Nadu, India
Job DescriptionWe are seeking a highly skilled C++ Python Developer with a strong background in software development, scripting, and EDA tool integration. This role focuses on creating, enhancing, and maintaining tools used in silicon design and ver...
1 Day ago
Business Analyst
Information Technology
- Chennai, Tamil Nadu, India
Project Role : Business AnalystProject Role Description : Analyze an organization and design its processes and systems, assessing the business model and its integration with technology. Assess current state, identify customer requirements, and defin...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in