Senior Data Engineer

Senior Data Engineer

Job Summary

Develop, maintain, and analyze datasets from diverse sources, including mobile and web, government agencies, web crawls, social media, and proprietary datasets, to create insights for our clients, power our platform, and create an innovative market understanding. Create designs and share ideas for creating and improving data pipelines and tools. This role will support maintaining our existing data pipelines and building new pipelines for increased customer insights.

Key Responsibilities

• Collaborate with cross-functional teams to identify and design requirements for advanced systems with respect to processing, analyzing, searching, visualizing, developing, and testing vast datasets to ensure data accuracy.
• Implement business requirements by collaborating with stakeholders.
• Become familiar with existing application code and achieve a complete understanding of how the applications function.
• Maintain data quality by writing validation tests.
• Understand variety of unique data sources.
• Create and document data documentation, including, processing systems and flow diagrams.
• Help maintain existing systems, including troubleshooting and resolving alerts.
• Expected to meet critical project deadlines.
• Excellent organizational, analytical, and decision-making skills.
• Excellent verbal, written, and interpersonal communication skills.
• Capable of working collaboratively and independently.
• Share ideas across teams to spread awareness and use of frameworks and tooling.
• Show an ownership mindset in everything you do; be a problem solver, be curious and be inspired to take action, be proactive, seek ways to collaborate and connect with people and teams in support of driving success.
• Continuous growth mindset, keep learning through social experiences and relationships with stakeholders, experts, colleagues and mentors as well as widen and broaden your competencies through structural courses and programs.

Skills/Experience/Certifications

• 8+ years experience in a data analysis, visualization, and manipulation.
• Extensive experience working with GCP services, including Big Query, Dataflow, Pub/Sub, Cloud Storage, Cloud Run, Cloud Functions and related technologies.
• Extensive experience with SQL and relational databases, including optimization and design.
• Experience with Amazon Web Services (EC2, RDS, S3, Redshift, EMR, and more).
• Experience with OS level scripting (bash, sed, awk, grep, etc.).
• Experience in AdTech, web cookies, and online advertising technologies is a plus.
• Testable and efficient Python coding for data processing and analysis.
• Familiarity with parallelization of applications on a single machine and across a network of machines.
• Expertise in containerized infrastructure and CI/CD systems, including CloudBuild, Docker, Kubernetes, Harness, and GitHub Actions.
• Experience with version control tools such as GIT, Github, and BitBucket.
• Experience with Agile Project Management tools such as Jira and Confluence.
• Experience with object-oriented programming, functional programming a plus.
• Analytic tools and ETL/ELT/data pipeline frameworks a plus.
• Experience with data visualization tools like Looker, Tableau, or Power BI.
• Experience working with global remote teams.
• Knowledge of data transformation processes.
• Google Cloud certification a plus.
• Proficiency in Microsoft Office Suite.
• Fluency in English and languages relevant to the team.