Lead Software Engineer - Bigdata

Overview

Reference 25000MUX

Responsibilities

We are seeking an experienced Data Engineer with strong SPARK, SACALA to join our data engineering team. The ideal candidate will have a solid background in data ingestion, data processing, and data management using technologies such as Spark & SCALA. Knowledge on SNOFLAKE & DBT are nice to have.

Knowledge on Apache Spark, Hive, and HDFS would be must have.. You will be responsible for migrating and testing data pipelines and ensuring efficient data storage and processing. In addition, you will work with CI/CD tools, such as Jenkins, and schedulers like CTRL-M, to automate data workflows.

Responsibilities

• Will work as an Individual contributor on Big Data platform delivering projects as mandated by business
• Perform tasks related to migrating & testing applications on Snowflake, Snowpark, Apache Spark, Hive , on new retail banking data Lakehouse.
• Good to have knowledge on DBT.
• Perform extensive testing on the developed code, debugging & fixing issues.
• Manage DevOPS as required & defined on the project
• Automate processes wherever required to improve efficiency of testing team.
• Co-ordinate with diverse geographically located teams to complete assigned activities.
• Participate in daily various agile ceremonies during the ongoing sprint and complete tasks as required.
• Conduct bug-free release validations and produce metrics, tests, and defect reports.
• Assist in developing and enforcing team guidelines.
• Being spoc for the team and provide information/status on ongoing activities on regular basis to management whenever needed.
  
  

• At least 5 to 7 Years of experience on big data platform and at least 4 years of extensive experience on application support and maintenance activities
• Proven experience with cloud platforms preferable Azure particularly on data services
• Good understanding of distributed computing frameworks like Apache Spark, Hadoop etc
• Perform Analysis on existing data storage systems Big Data & development of data solutions in Spark and SCALA
• High Proficiency in SQL and the following languages Scala / Python
• Nice to have experience in working on migrating data from on-premise databases preferably Big Data platform to Snowflake
• Expertise in building robust ELT/ETL processes , performance tuning of the data pipelines in Snowflake and should be able to trouble shoot the issues quickly
• Strong Knowledge on Integration concepts and design best practices
• Data Modeling & data integration, Advanced SQL skills for analysis, standardizing queries
• Proven experience in managing and mentoring data engineering teams
• Excellent interpersonal skills, with the ability to work across teams and communicate effectively with technical and non-technical stake holders
• Strong analytical and trouble shooting skills with proven ability to find solutions in complex data environments
  
  

We are committed to creating a diverse environment and are proud to be an equal opportunity employer. All qualified applicants receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status

Business insight

At Société Générale, we are convinced that people are drivers of change, and that the world of tomorrow will be shaped by all their initiatives, from the smallest to the most ambitious. Whether you’re joining us for a period of months, years or your entire career, together we can have a positive impact on the future. Creating, daring, innovating, and taking action are part of our DNA. If you too want to be directly involved, grow in a stimulating and caring environment, feel useful on a daily basis and develop or strengthen your expertise, you will feel right at home with us!

Still hesitating?

You should know that our employees can dedicate several days per year to solidarity actions during their working hours, including sponsoring people struggling with their orientation or professional integration, participating in the financial education of young apprentices, and sharing their skills with charities. There are many ways to get involved.

We are committed to support accelerating our Group’s ESG strategy by implementing ESG principles in all our activities and policies. They are translated in our business activity (ESG assessment, reporting, project management or IT activities), our work environment and in our responsible practices for environment protection.

Diversity and Inclusion

We are an equal opportunities employer and we are proud to make diversity a strength for our company. Societe Generale is committed to recognizing and promoting all talents, regardless of their beliefs, age, disability, parental status, ethnic origin, nationality, gender identity, sexual orientation, membership of a political, religious, trade union or minority organisation, or any other characteristic that could be subject to discrimination.