Senior Data Engineer

Annalect is currently seeking a back-end developer to join our Technology team. In this role, you will help grow our microservices and API layer which sit atop our Big Data infrastructure. We are passionate about building distributed back-end systems in a modular and reusable way. We're looking for people who have a shared passion for data and desire to build cool, maintainable and high-quality applications to use this data. In this role you will participate in shaping our technical architecture, design and development of software products, collaborate with back-end developers from other tracks, research and evaluation of new technical solutions, and helping to elevate the skills of more junior developers.

• Steward data and compute environments to facilitate usage of data assets
• Design, build, test and deploy scalable and reusable systems that handle large amounts of data
• Manage small team of developers
• Perform code reviews and provide leadership and guidance to junior developers
• Learn and teach new technologies
• Experience designing and managing data flows
• Experience designing systems and APIs to integrate data into applications
• 8+ years of Linux, Bash, Python, and SQL experience
• 4+ years using Spark and other Hadoop ecosystem software
• 4+ years using AWS cloud services, esp. EMR, Glue, Athena, and Redshift
• 4+ years managing team of developers
• Passion for Technology: Excitement for new technology, bleeding edge applications, and a positive attitude towards solving real world challenges

• BS, MS or PhD in Computer Science, Engineering, or equivalent real-world experience (You've learned something to be able to claim you are an engineer)
• Significant experience with Python, C++, or other popular language
• Experience with big data and/or infrastructure. Bonus for having experience in setting up Petabytes of data so they can be easily accessed. Understanding of data organization, i.e., partitioning, clustering, file sizes, file formats. Data cataloging with Hive/Hive meta store or Glue or something similar.
• Experience working with relational databases.
• Experience with Hadoop, Hive, Spark, or other data processing tools
• Experience in any of the modern data stack technologies like dbt, airbyte is desired.
• Experience building scalable data pipelines (Airflow experience a plus)
• Significant experience working with AWS and/or GCP
• Proven ability to independently execute projects from concept to implementation to launch and to maintain a live product