Senior Data Engineer

Job Description

• Help design, build and continuously improve the clients online platform.
• Research, suggest and implement new technology solutions following best practices/standards.
• Take responsibility for the resiliency and availability of different products.
• Be a productive member of the team.
  
  

Requirements

Key Responsibilities:

• Analyze and document current data integration approaches, including FTP/PowerShell scripts, InfoHub SQL, and Azure Data Factory (ADF).
• Map and document data flow dependencies, transformation logic, and data consumption patterns in Databricks and Power BI.
• Design and implement standardized, scalable data pipelines using Azure Data Factory.
• Develop efficient ingestion mechanisms for SAP (Vision), including support for future enhancements such as incremental or event-driven data loading.
• Review and refine transformation logic, promoting architectural best practices such as the medallion (bronze/silver/gold) model.
• Implement and maintain incremental data load strategies and Change Data Capture (CDC) processes.
• Develop robust monitoring, logging, and alerting systems to ensure data pipeline reliability and performance.
  
  

Required Qualifications:

• 5–7 years of hands-on data engineering experience, with 2–3 years specifically using Azure Databricks.
• Advanced SQL skills, including complex joins, common table expressions (CTEs), and analytical functions.
• Proven experience in ETL/ELT development using Databricks with PySpark, SQL, or Scala.
• Strong understanding of Delta Lake features such as ACID transactions, time travel, version control, and schema evolution.
• Experience integrating SAP data using ADF, APIs, or CDC strategies.
• Solid grasp of dimensional data modeling and medallion architecture principles.
• Proficiency with Azure services such as Azure Data Lake Storage (ADLS), Azure Data Factory, and Azure Key Vault.
• Familiarity with Git-based CI/CD processes and collaborative development using tools like Databricks Repos.
• Experience implementing data quality validation, logging, and monitoring frameworks.
  
  

Benefits

• A challenging, innovating environment.
• Opportunities for learning where needed.
  
  

Requirements

We are looking for a Technical Lead with 6 to 9 years of experience to join our team. The ideal candidate will possess expertise in JBoss J2EE, Spring Boot, Microservices, Spring Core, REST Web Services, Core Java, and PL/SQL. Experience in the Property & Casualty Insurance domain is essential. This is a hybrid role with day shifts, and no travel is required. Responsibilities: Lead the design and development of software solutions utilizing JBoss J2EE and Spring Boot. Oversee the implementation of Microservices architecture to ensure scalability and efficiency. Provide technical guidance and mentorship to junior developers. Develop and maintain REST Web Services that support various business processes. Use Core Java and PL/SQL to build reliable and efficient backend systems. Collaborate with cross-functional teams to gather, analyze, and define requirements. Ensure software solutions adhere to industry best practices and standards. Conduct code reviews to ensure consistency and high code quality. Troubleshoot and resolve technical issues promptly. Participate in the complete software development lifecycle, from concept to deployment. Implement and maintain CI/CD (Continuous Integration/Continuous Deployment) pipelines. Stay updated on the latest industry trends and technologies to keep our solutions innovative. Share knowledge and expertise to contribute to the team’s overall success. Qualifications: Strong experience with JBoss J2EE and Spring Boot. Expertise in Microservices architecture and REST Web Services. Proficiency in Core Java and PL/SQL. Prior experience in the Property & Casualty Insurance domain is mandatory. Experience with CI/CD pipelines and DevOps practices is a plus. Familiarity with Agile methodologies is a plus. Excellent problem-solving and analytical skills are desirable.