Big Data Engineer Consultant II/Senior Consultant I

At Allstate, great things happen when our people work together to protect families and their belongings from life’s uncertainties. And for more than 90 years our innovative drive has kept us a step ahead of our customers’ evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection.

Job Description

This role is responsible for executing multiple tracks of work to deliver Big Data solutions enabling advanced data science and analytics. This includes working with the team on new Big Data systems for analyzing data; the coding & development of advanced analytics solutions to make/optimize business decisions and processes; integrating new tools to improve descriptive, predictive, and prescriptive analytics. This role contributes to the structured and unstructured Big Data / Data Science tools of Allstate from traditional to emerging analytics technologies and methods. The role is responsible for assisting in the selection and development of other team members.

Job Responsibilities

As a data engineer, you will be responsible for the design, development and implementation of ETL pipeline. You will partner with Data Scientists as well as product stakeholders to create an advanced analytics eco-system.

• Obtain an overall understanding of data and processes
• Partner with the Product Owner on requirements and brainstorm implementation options
• Develop and deliver solutions using Databricks or other Spark tools and technologies
• Collaborate effectively with other teams (internal and external) in Hybrid model of work
• Learn new tools and implement continuous improvements in support of the Product’s strategic vision
• Evaluate detailed business, functional, and high-level technical requirements including recovery, security, and audit.
• Design detailed technical components with high-level architecture
• Apply reusability and future state architectures.
• Participates and provides feedback in design reviews.
• Develop or confirm detailed project or system change estimates or project plans. Calibrates estimating factors for continuous process improvement.
• Create enhanced technical documentation and implement changes.
• Conduct timely structured code reviews to ensure standards and systems interoperability.
• Create accurate test plans, conditions, and data. Participate in testing reviews.
• Conduct basic levels of module and integration testing according to process standards.
• Assists Quality Control (QC) personnel with functional tests.
• Identify support issues and initiate action to resolve the problem.
• Provide technical assistance.
• Analyze and resolve moderate problems.
• Execute change management activities supporting production deployment to Developers, Quality Control Analysts, and Environment Management personnel.
• Completes and delivers migration or change management form to above parties.
  
  

Primary Skills

Databricks, Spark, Scala, Shell scripting, SQL

Excellent communication skills

Ability to work in a high-paced team environment

Secondary Skills

AbInitio, MongoDB, Dremio, pyspark

ETL rewrite

Experience

4 to 6 Years of working experience

Bachelor’s degree / University degree in Engineering / Technology (B.Tech / B.E. / BCA)

Shift Timing

1 PM to 9:30 PM IST or as per project requirement

Primary Skills

Big Data Engineering, Big Data Systems, Big Data Technologies, Data Science, Influencing Others

Shift Time

Recruiter Info

Sunita M

maabd@allstate.com

About Allstate

Joining our team isn’t just a job — it’s an opportunity. One that takes your skills and pushes them to the next level. One that encourages you to challenge the status quo. One where you can shape the future of protection while supporting causes that mean the most to you. Joining our team means being part of something bigger – a winning team making a meaningful impact.

The Allstate Corporation is one of the largest publicly held insurance providers in the United States. Ranked No. 84 in the 2023 Fortune 500 list of the largest United States corporations by total revenue, The Allstate Corporation owns and operates 18 companies in the United States, Canada, Northern Ireland, and India. Allstate India Private Limited, also known as Allstate India, is a subsidiary of The Allstate Corporation. The India talent center was set up in 2012 and operates under the corporation's Good Hands promise. As it innovates operations and technology, Allstate India has evolved beyond its technology functions to be the critical strategic business services arm of the corporation. With offices in Bengaluru and Pune, the company offers expertise to the parent organization’s business areas including technology and innovation, accounting and imaging services, policy administration, transformation solution design and support services, transformation of property liability service design, global operations and integration, and training and transition.

Learn more about Allstate India here.