Overview
Title: Data Engineer
Location: Remote (India)
Job description:
Job Summary:
We are seeking a highly skilled and motivated Senior Data Engineer with 5+ years of experience to design, build, and maintain scalable data pipelines and systems. The ideal candidate will have strong hands-on experience with Fivetran, Airflow, SQL, Apache Spark, and AWS Cloud. Experience with Databricks is a strong plus.
Must-Have Qualifications:
* 5+ years of experience in data engineering or a similar role.
* Proficiency with Fivetran for connector-based data ingestion.
* Hands-on experience with Apache Airflow for workflow orchestration.
* Expert-level SQL skills for writing, tuning, and debugging complex queries.
* Strong experience with Apache Spark (PySpark or Scala).
* Deep understanding of AWS Cloud services and infrastructure relevant to data engineering.
* Strong problem-solving skills, attention to detail, and the ability to work independently.
Nice to Have:
* Experience with Databricks (development, cluster management, workspace integration).
* Familiarity with Delta Lake and data lakehouse architectures.
* Exposure to CI/CD tools and DataOps best practices.
Key Responsibilities:
* Design and implement efficient, scalable, and reliable data pipelines using Fivetran and Apache Airflow.
* Build ETL/ELT solutions to support analytics and business intelligence needs.
* Develop and optimize complex SQL queries for transformation and data quality validation.
* Utilize Apache Spark (batch and streaming) for large-scale data processing and transformation.
* Manage and deploy infrastructure and services in AWS Cloud (e.g., S3, Lambda, Glue, EMR, Redshift).
* Monitor, troubleshoot, and optimize data workflows to ensure high availability and data integrity.
* Collaborate with data analysts, scientists, and business stakeholders to define data requirements and deliver high-quality solutions.
* Implement best practices in data architecture, data security, and pipeline orchestration.
Job Type: Full-time
Work Location: Remote