Data Engineer

Overview

Role: Data Engineer
Years of Experience: 7+ Years of Experience
Location: Remote
Notice Period: Immediate to 30 Days (We will not be considering profiles beyond 30 Days Notice Period)

Mandaate Experitse:

• Python
• PySpark
• SQL / Queries
• AWS Elastic MapReduce (EMR)
• Amazon Managed Workflow for Apache Airflow (MWAA)\
• AWS CDK, Cloud-formation, Lambda, Step-funtion
• Athena
• Amazon Redshift
• AWS Glue Catalog
• AWS Simple Storage Service (S3)
• CI/CD: Github Actions

Job Description

• Collaborate and partner with Business Analyst teams located in US and EMEA regions.
• Interface across our Business Analyst and Data Science teams.
• Play a key role in integrating new data sources into our data & analytical ecosystem over AWS cloud
• Implement data lake solutions while addressing common data concerns, such as data quality, data governance
• Set the standard for technical excellence as we move / build our data ecosystem into the cloud
• understanding their common data problems and deliver scalable solutions

Responsibilities and Duties:

• 7-10 years of strong development experience performing ETL and/or data pipeline implementations.
• Expert in programming languages, preferably Python.
• Expert in delivering end-to-end analytic solutions using AWS services (EMR, Airflow, S3, Athena, Kinesis, Redshift).
• Experience in batch technologies like Hadoop, Hive, Athena, Presto.
• Strong SQL skills, including query optimization, schema design, complex analytics.
• Expert in data modeling and metadata management like Glue Catalog etc.
• Experience in deployment tools like GitHub actions, Jenkins, AWS Code Pipeline etc.
• Experience in data quality tools like Deque or Great Expectations is Nice To Have.

Job Types: Full-time, Permanent

Pay: ₹4,500,000.00 - ₹5,000,000.00 per year

Benefits:

• Health insurance
• Provident Fund

Schedule:

• Day shift
• Monday to Friday
• Morning shift

Supplemental Pay:

• Performance bonus

Experience:

• ETL: 7 years (Required)
• Python: 5 years (Required)
• Big data: 5 years (Required)
• AWS: 6 years (Required)
• AWS Glue: 5 years (Required)
• EMR: 5 years (Required)
• AWS CloudFormation: 5 years (Preferred)
• AWS MWAA: 5 years (Required)
• Athena: 4 years (Required)
• AWS S3: 4 years (Required)
• AWS CDK: 4 years (Required)

Work Location: Remote