Data Engineer(Databricks) Analyst

Job Title – Databricks Data Engineer - Analyst - Level 11 - ACS Song

Management Level: Level 11 – Analyst

Location: Kochi

Must have skills: Databricks

Good to have skills:

Experience: 1.5-3 years of experience is required

Educational Qualification: Graduation (Accurate educational details should capture)

Job Summary

We are seeking a Junior Data Engineer specializing in Databricks with 2–3 years of experience in data engineering, data pipelines, SQL development, and cloud-based data platforms. The role will focus on supporting the development, maintenance, and optimization of data pipelines and data transformation workflows using Databricks, PySpark, SQL, and Delta Lake. The ideal candidate should have a good foundation in Python, Spark concepts, ETL/ELT processes, and data warehouse or lakehouse architecture. This position requires collaboration with senior data engineers, architects, analysts, and business stakeholders to deliver reliable, scalable, and well-structured data solutions.

Roles and Responsibilities

• Support the design, development, and maintenance of data pipelines using Databricks, PySpark, SQL, and Delta Lake.
• Assist in building reliable data transformation workflows for reporting, analytics, and downstream business use cases.
• Work with senior team members to improve data quality, pipeline performance, documentation, and production stability.
• Develop strong hands-on expertise in Databricks, Spark-based processing, cloud data platforms, and modern data engineering practices.
• Develop and maintain data pipelines and transformation workflows using Databricks notebooks, jobs, workflows, PySpark, and SQL.
• Support ETL/ELT processes for ingesting, transforming, and loading structured and semi-structured data from multiple sources.
• Work with Delta Lake tables for data storage, updates, optimization, and reliable data processing.
• Assist in designing and maintaining data models, curated datasets, and lakehouse layers such as bronze, silver, and gold.
• Perform data validation, data quality checks, and basic troubleshooting of pipeline or data issues.
• Optimize Spark SQL, PySpark jobs, and Databricks workflows for performance and reliability under guidance from senior engineers.
• Collaborate with data engineers, analysts, and business users to understand requirements and deliver data solutions.
• Follow best practices for version control, documentation, testing, deployment, monitoring, and production support.

Professional and Technical Skills:

• 2–3 years of experience in data engineering, SQL development, ETL/ELT development, or cloud-based data processing.
• Hands-on experience working with Databricks in development or production environments.
• Experience supporting data pipelines, data transformations, analytics datasets, or lakehouse-based data solutions.
• Good programming experience in Python and hands-on exposure to PySpark for data processing.
• Strong SQL skills, including joins, aggregations, CTEs, window functions, data validation, and query troubleshooting.
• Good understanding of Databricks concepts such as notebooks, clusters, jobs, workflows, Delta tables, and basic performance tuning.
• Good understanding of Delta Lake, data lakehouse architecture, ETL/ELT workflows, and medallion architecture.
• Exposure to cloud platforms such as Azure, AWS, or GCP and cloud storage services is preferred.
• Familiarity with Git, Airflow, dbt, CI/CD, or BI/reporting tools is an added advantage.

Additional Information

Behavioral and Collaboration Skills

• Strong analytical and problem-solving skills.
• Good communication skills with the ability to work with technical and business teams.
• Eagerness to learn, take feedback, and grow under the guidance of senior engineers.
• Detail-oriented approach with a focus on data accuracy, reliability, and timely delivery.

About Our Company | Accenture (do not remove the hyperlink)

• 2–3 years of experience in data engineering, SQL development, ETL/ELT development, or cloud-based data processing.
• Hands-on experience working with Databricks in development or production environments.
• Experience supporting data pipelines, data transformations, analytics datasets, or lakehouse-based data solutions.
• Good programming experience in Python and hands-on exposure to PySpark for data processing.
• Strong SQL skills, including joins, aggregations, CTEs, window functions, data validation, and query troubleshooting.
• Good understanding of Databricks concepts such as notebooks, clusters, jobs, workflows, Delta tables, and basic performance tuning.
• Good understanding of Delta Lake, data lakehouse architecture, ETL/ELT workflows, and medallion architecture.
• Exposure to cloud platforms such as Azure, AWS, or GCP and cloud storage services is preferred.

Familiarity with Git, Airflow, dbt, CI/CD, or BI