GCP Data Engineer

Overview

JOB DESCRIPTION:

Open Position – Senior Data Engineer

Location – Whitefield, Bengaluru (Hybrid)

Experience – 4+ Years

Must-Have: Python, SQL, GCP and 2+ years of experience in a customer-facing role.

Responsibilities:

● Design, build, and maintain robust data pipelines and ETL processes using PySpark and other big data technologies.

● Optimize data processing workflows for cost-effectiveness, performance, and scalability.

● Implement data engineering best practices for data quality, reliability, and accessibility.

● Deploy and manage data solutions on cloud platforms (AWS, Azure, GCP), ensuring security, scalability, and compliance.

● Develop and automate logging, monitoring, and alerting mechanisms to ensure data pipeline health and performance.

● Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.

● Prototype and implement new technologies and tools to enhance data infrastructure and analytics capabilities.

Qualifications:

● Bachelor’s degree in Computer Science, Engineering, or a related field.

● Minimum 4+ years of experience programming and developing applications using Python.

● Expertise in distributed computing frameworks such as PySpark for big data processing.

● Strong understanding of cloud platforms (AWS, Azure, GCP) and their services for data storage, compute, and analytics.

● Experience with designing and optimizing cost-effective and performance-improved data pipelines.

● Proficiency in SQL and working with both relational(PostgreSQL, MySQL, Microsoft SQL Server) and non-relational databases(MongoDB, Cassandra, Redis).

● Familiarity with containerization (Docker, Kubernetes) and orchestration tools.

● Experience with logging, monitoring, and alerting tools (e.g., ELK Stack, Prometheus, Grafana).

● Excellent problem-solving skills with the ability to analyze complex data issues and provide solutions efficiently.

Preferred Skills:

● Knowledge of streaming data processing frameworks (e.g., Apache Kafka, Spark Streaming).

● Familiarity with machine learning frameworks and deployment (e.g., TensorFlow, PyTorch).

Google Cloud Professional Cloud Architect).

Job Type: Full-time

Work Location: In person