Free cookie consent management tool by TermsFeed Data Engineer | Antal Tech Jobs
Back to Jobs
3 Days ago

Data Engineer

decor
Gurugram, Haryana, India
Space Exploration & Research, Information Technology
Full-Time
McCain Foods Limited

Overview

Position Title: Data Engineer

Position Type: Regular - Full-Time

Position Location: Gurgaon

Requisition ID: 37277


Position Summary


Data engineers are mainly responsible for designing, building, managing, and operationalizing data pipelines to support key data and analytics use cases. They play a crucial role in constructing and maintaining a modern, scalable data platform that utilizes the full capabilities of a Lakehouse Platform.


You will be a key contributor to our data-driven organization, playing a vital role in both building a modern data platform and maintaining our Enterprise Data Warehouse (EDW). You will leverage your expertise in the Lakehouse Platform to design, develop, and deploy scalable data pipelines using modern and evolving technologies. Simultaneously, you will take ownership of the EDW architecture, ensuring its performance, scalability, and alignment with evolving business needs. Your responsibilities will encompass the full data lifecycle, from ingestion and transformation to delivery of high-quality datasets that empower analytics and decision-making.


Duties and responsibilities


Build data pipelines using Azure Databricks:

  • Build and maintain scalable data pipelines and workflows within the Lakehouse environment.
  • Transform, cleanse, and aggregate data using Spark SQL or PySpark.
  • Optimize Spark jobs for performance, cost efficiency, and reliability.
  • Develop and manage Lakehouse tables for efficient data storage and versioning.
  • Utilize notebooks for interactive data exploration, analysis, and development.
  • Implement data quality checks and monitoring to ensure accuracy and reliability.


Drive Automation:

  • Implement automated data ingestion processes using functionality available in the data platform, optimizing for performance and minimizing manual intervention.
  • Design and implement end-to-end data pipelines, incorporating transformations, data quality checks, and monitoring.
  • Utilize CI/CD tools (Azure DevOps/GitHub Actions) to automate pipeline testing, deployment, and version control.


Enterprise Data Warehouse (EDW) Management:

  • Create and maintain data models, schemas, and documentation for the EDW.
  • Collaborate with data analysts, data scientists and business stakeholders to gather requirements, design data marts, and provide support for reporting and analytics initiatives.
  • Troubleshoot and resolve any issues related to data loading, transformation, or access within the EDW.

Educate and train: The data engineer should be curious and knowledgeable about new data initiatives and how to address them. This includes applying their data and/or domain understanding in addressing new data requirements. They will also be responsible for proposing appropriate (and innovative) data ingestion, preparation, integration and operationalization techniques in addressing these data requirements. The data engineer will be required to train counterparts in these data pipelining and preparation techniques.

Ensure compliance with data governance and security: The data engineer is responsible for ensuring that the data sets provided to users are compliant with established governance and security policies. Data engineers should work with data governance and data security teams while creating new and maintaining existing data pipelines to guarantee alignment and compliance.


Qualifications


Education

Bachelor or master's in computer science, Information Management, Software Engineering, or equivalent work experience.


Work Experience

At least four years or more of working in data management disciplines including: data integration, modeling, optimization and data quality, and/or other areas directly relevant to data engineering responsibilities and tasks.

At least three years of experience working in cross-functional teams and collaborating with business stakeholders in support of a departmental and/or multi-departmental data management and analytics initiative.


Technical knowledge, Abilities, and skills

Ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata, and workload management. The ability to work with both IT and business in integrating analytics and data science output into business processes and workflows.

Strong knowledge of database programming languages and hands on experience with any RDBMS.


McCain Foods is an equal opportunity employer. As a global family-owned company, we strive to be the employer of choice in the diverse communities around the world in which we live and work. We recognize that inclusion drives our creativity, resilience, and success and makes our business stronger. All qualified applicants will receive consideration for employment without regard to race, religion, color, national origin, sex, age, veteran status, disability, or any other protected characteristic under applicable law.


McCain is an accessible employer. If you require an accommodation throughout the recruitment process (including alternate formats of materials or accessible meeting rooms), please
let us know and we will work with you to find appropriate solutions.


Your privacy is important to us. By submitting personal data or information to us, you agree this will be handled in accordance with McCain’s
Global Privacy Policy and Global Employee Privacy Policy, as applicable. You can understand how your personal information is being handled here.


Job Family:
Information Technology
Division: Global Digital Technology
Department: Global Data and Analytics
Location(s): IN - India : Haryana : Gurgaon

Company: McCain Foods(India) P Ltd

Share job
Similar Jobs
View All
1 Day ago
Machine Learning Engineer
Information Technology
  • 2 - 6 Yrs
  • Maharashtra
What you ll do: Lead ML model lifecycle, from research and experiments to implementation and deployment. Build and deploy deep learning models on GCP and edge devices , ensuring real-time inference. Combine multiple sensor in...
decor
1 Day ago
D-TechWorks - Java Full Stack Developer
Information Technology
Java Full stack - GurugramJob Title : Java Full stackExp : 5 to 7 yearsLocation : Gurgaon officeMandatory Skills : Java 8, Spring boot, Microservices, Reactjs, Database (MYSQL/PostgreSQL), Rest APIGood to have : AWS, Docker and orchestration to...
decor
1 Day ago
IT Manager (H/F) - AIRBUS INDIA PRIVATE LIMITED
Information Technology
  • Chennai, Tamil Nadu, India
Job Description: Description We are loo king for a highly customer focused, driven and dependable candidate to manage overall IT Operations at the Airbus India Training center (AITC) in Gurgaon. This position will functionally report to the Head of ...
decor
1 Day ago
PHP Developer
Information Technology
Job Description Youll Do Participating in the design and delivery of web-UI product. Developing solutions by designing system specifications and tests before delivering them. Identifying, analysing, and developing interfaces, flows and APIs. Int...
decor
1 Day ago
Technical Lead - .Net Core/AngularJS
Information Technology
  • Chennai, Tamil Nadu, India
We are seeking an experienced Technical Lead to guide a team of software developers in delivering robust and scalable enterprise solutions using ASP.NET Core, Angular, SQL Server, and Microsoft Azure.The ideal candidate will be responsible for end-t...
decor
1 Day ago
Business Analyst - Wealth Management
Information Technology
  • Chennai, Tamil Nadu, India
Primary skills:Domain->Capital Markets->Wealth Management A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to get to the heart of customer issues, diagnose problem areas, design innovative solutio...
decor
1 Day ago
Sr. Software Engineer - Pentaho Data Integration Job
Information Technology
  • Chennai, Tamil Nadu, India
We use cookies to offer you the best possible website experience. Your cookie preferences will be stored in your browser’s local storage. This includes cookies necessary for the website's operation. Additionally, you can freely decide and change any...
decor
1 Day ago
Dataviv Technologies - Quality Assurance Test Engineer - Manual & Automation Testing
Information Technology
  • Chennai, Tamil Nadu, India
Job Title: QA Tester (Manual & GurugramEmployment Type: Full-timeJob SummaryWe are looking for a skilled and detail-oriented QA Tester who is proficient in both manual and automation testing. The ideal candidate will be responsible for ensuring the ...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media