Lead Data Engineer

ABOUT US

We are Alter Domus. Meaning “The Other House” in Latin, Alter Domus is proud to be home to 85% of the top 30 asset managers in the alternatives industry, and more than 5,000 professionals across 23 countries.

With a deep understanding of what it takes to succeed in alternatives, we believe in being different. Invest yourself in the alternative, and join an organization where you progress on merit, where you can speak openly with whoever you are speaking to, and where you will be supported along whichever path you choose to take.

Find out more about life at Alter Domus at careers.alterdomus.com

We are looking for a skilled Lead Data Engineer to join our growing team. You will play a pivotal role in designing, building, and maintaining our data infrastructure and pipelines. Working closely with data analysts, data scientists, and various business stakeholders, you’ll ensure data is reliable, efficient, and accessible in a scalable manner.

Key Responsibilities:

• Data Pipeline Orchestration
  • Design, build, and maintain end-to-end data pipelines using Airflow (including managed services like Amazon MWAA) to orchestrate, schedule, and monitor batch/streaming workflows.
  • Implement DAGs (Directed Acyclic Graphs) with retry logic, error handling, and alerting to ensure data quality and pipeline reliability.
• Data Ingestion & Transformation
  • Integrate data from various sources using Airbyte for ingestion and dbt for transformations in a scalable and modular fashion.
  • Collaborate with Data Analysts and Data Scientists to implement transformations and business logic, ensuring data is analytics-ready.
• Data Modeling & Warehousing
  • Design and implement efficient data models for both structured and semi-structured data in AWS S3 (data lake) and Snowflake (data warehouse).
  • Ensure data schemas and transformations support advanced analytics, BI reporting, and machine learning use cases.
• Data Governance & Security
  • Utilize AWS Lake Formation APIs and best practices to maintain data security, access controls, and compliance.
  • Work closely with IT security to establish robust encryption standards, audit trails, and identity/role-based access.
• Performance Optimization
  • Optimize AWS Athena queries and configurations (e.g., data partitioning) for performance and cost efficiency.
  • Monitor and tune Airflow DAGs, Snowflake queries, and data transformations to improve throughput and reliability.
• Collaboration & Stakeholder Management
  • Partner with cross-functional teams, including DevOps, Platform Engineering, and Data Science, to ensure seamless integration of data workflows and systems.
  • Communicate technical solutions effectively to non-technical stakeholders and leadership, translating requirements into actionable tasks.
• Continuous Improvement
  • Participate in architecture reviews, code reviews, and troubleshooting sessions to ensure quality and alignment with best practices.
  • Remain current with emerging trends in data engineering, orchestration tools (Airflow, MWAA), and cloud services (AWS, Snowflake).

Qualifications & Skills:

• Education & Experience
  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • 10+ years of experience as a Data Engineer or in a similar role working with cloud-based data platform

Technical Skills:

• Cloud & Orchestration:
  • Airflow (self-managed or managed services like Amazon MWAA) for workflow orchestration, DAG development, and scheduling.
  • Familiarity with best practices for Airflow DAG structure, dependency management, and error handling.
• AWS Expertise:
  • Hands-on experience with AWS Lake Formation, S3, Athena, and related services (e.g., Lambda, Glue, IAM).
• Snowflake:
  • Proficient in setting up data warehouses, configuring security, and optimizing queries on Snowflake.
• Data Ingestion & Transformation:
  • Experience with Airbyte or similar tools for data ingestion.
  • dbt or other SQL-based transformation frameworks for modular data processing.
• Programming:
  • Proficiency in Python and/or Java/Scala for building data pipelines and custom integrations.
• Query Languages:
  • Advanced knowledge of SQL for data manipulation and analysis.
• Soft Skills
  • Strong problem-solving and analytical abilities.
  • Excellent communication and collaboration skills, able to effectively work in cross-functional teams.
  • Ability to operate in a fast-paced, agile environment and manage multiple priorities simultaneously

Preferred Certifications & Experience

• AWS certifications (e.g., AWS Certified Data Analytics – Specialty, AWS Certified Solutions Architect) are a plus.
• Experience with CI/CD pipelines, containerization (Docker, Kubernetes), and infrastructure as code (Terraform, CloudFormation) is beneficial.
• Familiarity with DevOps best practices for managing Airflow environments (e.g., version control for DAGs, automated testing, monitoring).
While familiarity with financial services data especially private equity and alternative investments is not necessary, it will be highly impactful for this role.

WHAT WE OFFER:

We are committed to supporting your development, advancing your career, and providing benefits that matter to you.

Our industry-leading Alter Domus Academy offers six learning zones for every stage of your career, with resources tailored to your ambitions and resources from LinkedIn Learning.

Our global benefits also include:

• Support for professional accreditations such as ACCA and study leave
• Flexible arrangements, generous holidays, birthday leave
• Continuous mentoring along your career progression
• Active sports, events and social committees across our offices
• Support with mental, physical, emotional and financial support 24/7 from our Employee Assistance Program
• The opportunity to invest in our growth and success through our Employee Share Plan
• Plus additional local benefits depending on your location

Equity in every sense of the word

We are in the business of equity, in every sense of the word. For us, this means taking action to ensure every colleague has equal opportunity, valuing every voice and experience across our organisation, maintaining an inclusive culture where you can bring your whole self to work, and making Alter Domus a workplace where everyone feels they belong.

We celebrate our differences, and understand that our success relies on diverse perspectives and experiences, working towards shared goals and a common purpose. Thanks to the work of our Group DE&I Committee and network of DE&I Champions, we empower all of our people to be truly invested in the alternative.

We are committed to ensuring an inclusive recruiting and onboarding process. Please contact our hiring team if you require any accommodations to make our recruitment process more accessible for you.

(Alter Domus Privacy notice can be reviewed via Alter Domus webpage: https://alterdomus.com/privacy-notice/)

#LI-HYBRID