Cloud Engineer - II

Overview

• Showcase strong analytical and problem-solving capabilities by consulting with clients and other stakeholders to understand business problems.
• Responsible for designing, storing, processing, and maintaining of large-scale data and related infrastructure.
• Is a developer who is responsible to build data pipelines, implement data quality, build data model, monitor, and optimize data pipelines to minimize runtime, and adhere to customer and MathCo. set standards.
• Has strong conceptual understanding of Data Warehousing and ETL, Data Governance and Security, Cloud Computing, and Batch & Real Time data processing.
• Has strong execution knowledge of Data Modeling, Databases in general (SQL and NoSQL), software development lifecycle and practices, unit testing, functional programming, etc.
• Understanding of Medallion architecture pattern.
• Has worked on at least one cloud platform.
• Has executed multiple end-to-end data engineering projects.
• Manages conversation with the client stakeholders to understand the requirement and translate it into technical outcomes.

Required Tech Stack

• Strong proficiency in SQL.
• Experience working on any of the three major cloud platforms i.e., AWS/Azure/GCP.
• Working knowledge of an ETL and/or orchestration tools like IICS, Talend, Matillion, Airflow, Azure Data Factory, AWS Glue, GCP Composer, etc.
• Working knowledge of one or more OLTP databases (Postgres, MySQL, SQL Server, etc.).
• Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Azure Synapse, Hive, Big Query, etc.
• Proficient in at least one programming language used in data engineering, such as Python (or Scala/Rust/Java).
• Has strong execution knowledge of Data Modeling (star schema, snowflake schema, fact vs dimension tables).
• Proficient in Spark and related applications like Databricks, GCP DataProc, AWS Glue, EMR, etc.
• Has worked on code and SQL query optimization.
• Strong knowledge of version control systems like Git to manage source code repositories and designing CI/CD pipelines for continuous delivery.

• Understanding of code optimization and security best practices.
• Strong problem-solving skills.
• Able to analyze problems and quickly develop effective and actionable strategies.
• Proficient in written and verbal communication.
• Exercises independent judgment to determine the best course of action within established practices and procedures.