Senior Data Engineer

UWorld is a worldwide leader in online test prep for college entrance, undergraduate, graduate, and professional licensing exams throughout the United States. Since 2003, over 2 million students have trusted us to help them prepare for high-stakes examinations.

We are seeking a Senior Data Engineer who is passionate about creating an excellent user experience and enjoys taking on new challenges. The Data Engineer will be responsible for the design, development, testing, deployment, and support of our Data Analytics and the Data warehouse platform.

Requirement

Minimum Experience:

• Master's/bachelor’s degree in computer science or a related field.
• 6+ years of experience as a Data Engineer with experience in Data Analysis, ingestion, cleansing, validation, verification, and presentation (reports and dashboards).
• 4+ years of working knowledge and experience utilizing the following: Python, Spark/PySpark, Big Data Platforms (Data Bricks/Delta Lake), REST services, MS SQL Server/MySQL, MongoDB, and Azure Cloud.
• Experience with SQL, PL/SQL, and Relational Databases (MS SQL Server/MySQL/Oracle). Experience with Tableau/Power BI, NoSQL (MongoDB), and Kafka is a plus.
• Experience with REST API, Web Services, JSON, Build and Deployment pipelines (Maven, Ansible, Git), and Cloud environments (Azure, AWS, GCP) is desirable.

Job Responsibilities:

The software developer will perform the following duties:

• Understand data services and analytics needs across the organization and work on the data warehouse and reporting infrastructure to empower them with accurate information for decision-making.
• Develop and maintain a data warehouse that aggregates data from multiple content sources, including NoSQL DBs, RDBMS, Big Query, Salesforce, social media, other 3rd party web services (RESTful, JSON), flat-file stores, and application databases (OLTPs).
• Use Python, Spark/PySpark, Data Bricks, Delta Lake, SQL Server, Mongo DB, Jira, Git/Bit Bucket, Confluence, Data Bricks/Delta Lake, REST services, Tableau, Unix/Linux shell scripting, and Azure Cloud for data ingestion, processing, transformations, warehousing, and reporting.
• Develop scalable data pipelines using Data connectors, distributed processing transformations, schedulers, and data warehouse.
• Understanding of data structures, analytics, data modeling, and software architecture and applying this knowledge to problem solving.
• Develop, modify, and test algorithms that can be used in scripts to store, locate, cleanse, verify, validate, and retrieve specific documents, data, and information.
• Develop analytics to understand product sales, marketing impact, and application usage for UWorld products and applications.
• Employ best practices for code sharing and development to ensure common code base abstraction across all applications. Continuously be up-to-date on industry standard practices in big data and analytics and adopt solutions to the UWorld data warehousing platform.
• Work with QA engineers to ensure the quality and reliability of all reports, extracts, and dashboards by process of continuous improvement.
• Collaborate with technical architects, developers, subject matter experts, QA team, and customer care team to drive new enhancements or fix bugs promptly.
• Work in an agile environment such as Scrum.

Soft Skills.

• Working proficiency and communication skills in verbal and written English.
• Excellent attention to detail and organization skills and ability to articulate ideas clearly and concisely.
• Ability to work effectively within a changing environment that is going through high growth.
• Exceptional follow-through, personal drive, and ability to understand direction and feedback.
• Positive attitude with a willingness to put aside ego for the sake of what is best for the team.