Senior Data Engineer

Company Overview

Job Overview

Come join the Intuit Data platform (IDP) team as a Sr Software Engineer. The IDP team owns the Intuit Analytics Platform which is the foundation of big data at Intuit and which enables real-time data ingestion, cataloging, analytics and machine learning on all of Intuit’s data. With Intuit’s customers growing rapidly year over year, the volume of data we handle is ever increasing and the excellent data engineering we do in IDP helps Intuit keep up with this data volume and leverage it for machine learning and data-driven product innovations.

Responsibilities

• Design and build big data platform and Big Data solutions primarily based on open source technologies that is fault-tolerant & scalable.
• Build solid & scalable architecture to address on the Normalization, lineage, Data governance, Ontology typical data catalog UCs
• Design solutions that involve complex, multi-system integration, possibly across BUs or domains
• Work with Analysts and Data Scientists to identify datasets needed for deep customer insights and for building operational propensity models.
• Hands on contribution to biz logic using Hadoop echo system (Java MR, Hbase, Hive) and build framework(s) to support data pipelines on streaming applications
• Work on technologies related to NoSQL, SQL and InMemory platform(s)
• Conducts code reviews to ensure code quality, consistency and best practices adherence.
• Conducts quick Proof of Concept (POCs)for feasibility studies and take it to the prod
• Lead fast moving development teams using agile methodologies.
• Lead by example, demonstrating best practices for unit testing, CI/CD, performance testing, capacity planning, documentation, monitoring, alerting, and incident response.

Qualifications

• 8+ years of relevant experience with at least 5+ years on the big data domain
• Should have an experience in designing E2E ecosystem for big data and analytical platform.
• Good experience in building big data platform and Big Data solutions primarily based on Hadoop echo system that is fault-tolerant & scalable.
• Expert level experience with Java programming, RESTful web services and micro service architectures.
• In depth understanding in problem solving, data structure and Algorithms
• Expert level experience designing high throughput data services.
• Experience with Big Data Technologies (Hive, HBase, Spark, Kafka, flink, Storm, Map Reduce, HDFS, Splunk, zoo keeper, MemSQL, Cassandra, Redshift, Graphdb ), understands the concepts and technology ecosystem around both real-time and batch processing in Hadoop.
• Strong communication skills
• BE/Btech/MS in Computer Science (or equivalent)
• Employs effective listening and demonstrates strong collaboration to lead change by example and through influence