GCP Data Engineer

Job Description:

As a Senior Data Engineer with a focus on pipeline migration from SAS to Google Cloud Platform (GCP) technologies, you will tackle intricate problems and create value for our business by designing and deploying reliable, scalable solutions tailored to the company’s data landscape. You will lead the development of custom-built data pipelines on the GCP stack, ensuring seamless migration of existing SAS pipelines. Additionally, you will mentor junior engineers, define standards and best practices, and contribute to strategic planning for data initiatives.

Responsibilities:

● Lead the design, development, and implementation of data pipelines on the GCP

stack, with a focus on migrating existing pipelines from SAS to GCP

technologies.

● Develop modular and reusable code to support complex ingestion frameworks,

simplifying the process of loading data into data lakes or data warehouses from

multiple sources.

● Mentor and guide junior engineers, providing technical oversight and fostering

their professional growth.

● Work closely with analysts, architects, and business process owners to translate

business requirements into robust technical solutions.

● Utilize your coding expertise in scripting languages (Python, SQL, PySpark) to

extract, manipulate, and process data effectively.

● Leverage your expertise in various GCP technologies, including BigQuery,

Dataproc, GCP Workflows, Dataflow, Cloud Scheduler, Secret Manager, Batch,

Cloud Logging, Cloud SDK, Google Cloud Storage, IAM, and Vertex AI, to

enhance data warehousing solutions.

● Lead efforts to maintain high standards of development practices, including

technical design, solution development, systems configuration, testing,

documentation, issue identification, and resolution, writing clean, modular, and

sustainable code.

● Understand and implement CI/CD processes using tools like Pulumi, GitHub,

Cloud Build, Cloud SDK, and Docker.