Senior Data Engineer

Overview

We are seeking an experienced Data Engineer with 6–8 years of hands-on experience in building scalable data pipelines, modern data platforms, and end‑to‑end data engineering solutions. The ideal candidate will be strong in distributed data processing, cloud technologies, data modeling, and best practices in data quality, governance, and performance optimization.

• Design, build, and maintain scalable ETL/ELT pipelines using tools such as Spark, PySpark, Dataflow, Databricks, or similar.
• Develop batch and real‑time streaming pipelines leveraging Kafka, EventHub, Kinesis, or Pub/Sub.
• Optimize data ingestion, transformation, and storage for performance and scalability.

• Build and maintain robust data lake and data warehouse architectures.
• Work with modern cloud platforms such as Azure, AWS, or GCP to deploy end‑to‑end data engineering solutions.
• Implement data orchestration using Airflow, Azure Data Factory, Step Functions, or similar tools.

• Design conceptual, logical, and physical data models to support analytics and business intelligence.
• Work with storage technologies such as Delta Lake, Snowflake, BigQuery, Redshift, Synapse, or equivalent.

• Implement automated data validation, monitoring, and observability frameworks.
• Ensure data governance practices including metadata management, lineage, and cataloging.
• Enforce security standards related to PII, encryption, RBAC, and compliance requirements.

• Partner with data analysts, scientists, product teams, and business stakeholders to deliver data solutions.
• Provide technical leadership, code reviews, and mentorship to junior engineers.
• Translate business requirements into technical specifications.

• 6–8 years of professional experience as a Data Engineer.
• Strong programming skills in Python, Scala, or Java.
• Hands‑on experience with Apache Spark / PySpark.
• Solid understanding of SQL and experience with performance tuning.
• Proven experience building pipelines in Azure/AWS/GCP (any one cloud is fine).
• Experience with CI/CD tools (GitHub Actions, Azure DevOps, Jenkins, etc.).
• Knowledge of data warehousing principles, dimensional modeling, and lakehouse architectures.
• Experience with containerization (Docker, Kubernetes) is a plus.
• Familiarity with ML workflows and MLOps is a plus.

• Experience with dbt, Terraform, or infrastructure-as-code.
• Exposure to BI tools such as Power BI, Tableau, Looker.
• Understanding of microservices and distributed systems.

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Data Engineering, or a related field.