Senior Data Engineer

Overview

Job Title: Sr. Data Engineer

Experience: 5+ Years

Location: Bangalore (Hybrid)

Job Description:

We are looking for an experienced Sr. Data Engineer to join our team. The ideal candidate will have extensive experience in managing data pipelines, cloud platforms, and operational support, with a strong focus on automation, ETL processes, and troubleshooting.

Key Responsibilities:

• Automate processes using Python and shell scripting for task automation and system-level troubleshooting.
• Manage and optimize ETL processes using tools like Informatica Cloud.
• Utilize strong SQL skills to query data, create custom checks, and monitor services.
• Oversee and monitor data pipelines in upstream, batch, and streaming environments.
• Utilize pipeline orchestration tools such as Luigi or Airflow.
• Provide operational support, troubleshoot issues, and resolve incidents.
• Work with cloud platforms (AWS, GCP, Azure) and environments like OpenShift and Kubernetes.
• Set up monitoring dashboards and use tools like Prometheus and Grafana for service health and anomaly detection.
• Handle user access management, including permissions validation.
• Document and manage data flows and dependencies between different services.
• Follow DevOps and Agile practices to ensure smooth project delivery.

Must-Have Skills:

• Proficiency in Python and Shell scripting.
• Experience with ETL tools (e.g., Informatica Cloud) and data pipeline management.
• Strong SQL skills for data querying and creating custom checks.
• Experience with cloud platforms (AWS, GCP, Azure) and containerized environments (OpenShift, Kubernetes).
• Knowledge of monitoring tools like Prometheus, Grafana, and incident troubleshooting.
• Experience with user access management and data security practices.
• Familiarity with DevOps and Agile environments.

Nice-to-Have Skills:

• Strong communication skills and experience with collaboration tools like Google Docs and Jira.
• Experience in incident management and root cause analysis.
• Knowledge of Apache Kafka and streaming platforms.
• Data governance expertise.

Job Type: Contractual / Temporary

Pay: ?130,000.00 - ?150,000.00 per month

Schedule:

• Day shift

Work Location: In person