6860 - Quality Assurance Data Engineer

Quality Assurance Data Engineer – Data Platform Team (Databricks & Azure)
Expreince : 5 - 7 Years
Work location : Remote ( Must be flexible to travel to Bangalore / Hyd / Pune when needed )

New / Replacement - New
Job Description: Quality Assurance Data Engineer – Data Platform Team (Databricks & Azure)

Position Overview:
We are seeking a highly motivated and detail-oriented Quality Assurance (QA) Data Engineer to join our Data Platform Team. In this role, you will ensure the quality, reliability, and performance of our data platform by overseeing QA processes, executing manual and automated testing, and driving continuous improvements in testing methodologies. You will work closely with data engineers, product managers, and other stakeholders to ensure that our platform meets the highest standards of quality and performance.

Key Responsibilities:
1. Quality Assurance Processes and Coverage:

• Oversee and maintain the overall quality assurance processes, ensuring comprehensive test coverage across all features and fixes.
• Develop and document the testing strategy, including manual and automated testing approaches, to ensure consistent quality across releases.

Generate detailed reports summarizing the results of performance and load testing, and document findings in Confluence or other collaboration tools.

• Maintain comprehensive documentation of testing processes, test cases, and automation scripts.
• Conduct testing on Databricks Runtime Long Term Support (DBR LTS) versions to ensure stability and compatibility.

Collaborate with data engineers, product managers, and other stakeholders to understand requirements, conduct demo sessions and ensure alignment of quality goals.

2. Manual Testing:

• Execute manual quality assurance testing for new features, bug fixes, and enhancements.
• Perform regression testing before each new release to ensure existing functionality remains intact.
• Provide sign-off for new releases, ensuring they meet quality standards and business requirements.

3. Automation and Load Testing:

• Automate existing manual test cases to improve efficiency and reduce manual effort.
• Develop, maintain, and enhance automation test scripts for functional regression, performance, and load testing.
• Analyze automation test runs, identify failures, and create detailed bug reports for the engineering team.

Analyze performance testing results, identify bottlenecks, and provide actionable recommendations for improvement.

Required Qualifications:
Proven experience in quality assurance and testing for data platforms using Databricks and Azure - Must have
Strong understanding of QA methodologies, tools, and best practices
Hands-on experience with automation testing frameworks and tools, also hands on in manual testing
Experience with performance and load testing tools and techniques
Proficiency in programming languages such as Python or Scala for test automation
Familiarity with data engineering concepts, including ETL pipelines, data lakes, and data warehouses
Experience with version control systems (e.g., Git) and CI/CD pipelines
Strong analytical and problem-solving skills, with the ability to identify and resolve issues effectively
Excellent communication and collaboration skills

Preferred Qualifications:
Experience with Databricks-specific testing tools and frameworks
Knowledge of Azure services such as Azure Data Factory, Azure Batch, and SQL Datawarehouse
Familiarity with Confluence, JIRA, or similar collaboration and project management tools
Experience with monitoring and logging tools for performance analysis

Client interview or screening round - Yes