Software Engineer - Data Engineer

About Beyond Key

We are a Microsoft Gold Partner and a Great Place to Work-certified company. "Happy Team Members, Happy Clients" is a principle we hold dear. We are an international IT consulting and software services firm committed to providing. Cutting-edge services and products that satisfy our clients' global needs. Our company was established in 2005, and since then we've expanded our team by including more than 350+ Talented skilled software professionals. Our clients come from the United States, Canada, Europe, Australia, the Middle East, and India, and we create and design IT solutions for them. If you need any more details, you can get them at https://www.beyondkey.com/about.

Job Summary

We are looking for a Data Engineer with strong expertise in Databricks, PySpark, SQL, and Azure Data Factory (ADF) to design and optimize scalable data pipelines. Experience with Snowflake and DBT is a plus. The ideal candidate will have a proven track record in building efficient ETL/ELT processes, data warehousing, and cloud-based data solutions.

Key Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks.
• Process and transform large datasets using PySpark (DataFrames, Spark SQL, optimizations) in Databricks.
• Write and optimize complex SQL queries for data extraction, transformation, and loading.
• Implement data lakehouse architectures using Delta Lake in Databricks.
• (Optional) Manage and optimize Snowflake data warehouses (table structures, performance tuning).
• (Optional) Use DBT (Data Build Tool) for modular and scalable data transformations.
• Ensure data quality, governance, and monitoring across pipelines.
• Collaborate with data analysts and business teams to deliver actionable insights.
  
  

Qualifications

• 4+ years of hands-on experience in Databricks, PySpark, SQL, and ADF.
• Strong expertise in Databricks (Spark, Delta Lake, Notebooks, Job Scheduling).
• Proficiency in Azure Data Factory (ADF) for pipeline orchestration.
• Advanced SQL skills (query optimization, stored procedures, partitioning).
• Experience with data warehousing concepts (ETL vs. ELT, dimensional modeling).
• (Good to Have) Familiarity with Snowflake (warehouse management, Snowpipe).
• (Good to Have) Knowledge of DBT (Data Build Tool) for transformations.
• (Bonus) Python scripting for automation and data processing.
  
  

Preferred Qualifications

• Certifications (Databricks Certified Data Engineer, Azure Data Engineer).
• Experience with CI/CD pipelines (Azure DevOps, GitHub Actions).
• Knowledge of streaming data (Kafka, Event Hubs) is a plus.
  
  

Share with someone awesome

View all job openings