Overview
Role description
Role - GCP Data Engineer with Python/Pyspark
Experience: 3~9 years
Preferred - Data Engineering Background
Location - Bangalore
Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
Design, develop, and deploy scalable ETL/ELT pipelines using PySpark on GCP.
Utilize GCP services extensively, including BigQuery (data warehousing), Cloud Storage, Dataproc, and Dataflow.
Optimize PySpark jobs for performance and reliability, and fine-tune BigQuery queries.
Implement complex transformations and process large volumes of structured/unstructured data using Spark SQL and PySpark
Build and manage automated workflows using Apache Airflow or Cloud Composer.
Strong proficiency in Python and SQL is essential
Extensive hands-on experience with PySpark.
Proven experience with Google Cloud Platform (GCP) services.