Data Engineer - Sr. Consultant level (Hadoop, Spark, Python, GenAI, 11-15 years exp)

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

This is a great opportunity to work with Engineering and AI team to scale and structure large scale data engineering and AI/ML that drives significant revenue for Visa. As a member of the Post Purchase Solutions your role will involve developing and implementing practices that will allow deployment of machine learning models in large data science projects.

You must be a hands-on expert able to navigate both data engineering and data science disciplines to build effective engineering solutions that support AIML models. You will partner closely with global stakeholders across products in VERA including Data Science to help create and prioritize our strategic roadmap. You will then leverage your expert technical knowledge of data engineering, tools and data architecture in the design and creation of the solutions on our roadmap.

Essential Functions:

• You’ll be the face of the team representing us in various technical forums.
• You’ll help our customers achieve their business goals by understanding their requirements. You will build deep partnerships with product management to make sure that the products we are building deliver real value.
• You’ll analyze business requirements and architect highly secure, robust, and scalable solutions.
• You’ll drive internal proof of concept initiatives. When needed, quickly design and implement a prototype of a system or component with a proper architecture, and then lead or hand over to engineering team to implement.
• You’ll champion efforts to architect, design and implement various components of our global transaction processing systems.
• Your DNA will include following and creating software best practices and processes.
• You’ll coach and mentor your team members and help them grow professionally. You will also create an atmosphere of mutual accountability.
• You’ll play key role in meetings and discussions with cross-functional and non-technical teams.
• This is a hybrid position. Expectation of days in the office will be confirmed by your Hiring Manager.
  
  

Basic Qualifications:

• 11+ years of relevant work experience with a Bachelor’s Degree or with an Advanced degree
• Solid experience in Big Data engineering, with experience in Hadoop, Apache Spark, Python, and SQL.
• Proficiency in creating and managing large-scale data pipelines, handling complex, high-volume, multi-dimensional data, and machine learning models.
• Expertise in developing ETL processes, maintaining Spark pipelines.
• Expertise in Python and experience in REST APIs, container-based technologies, and implementing containerization strategies using Docker and Kubernetes.
• Proficient in popular technologies and cloud services, such as Kafka, Hazlecast, AWS services, and containerized environments (Docker, K8s).
• Skilled in Unix/Shell or Python scripting, using scheduling tools like Airflow and Control-M, and maintaining OPS tools.
• Familiarity with Agile development incorporating TDD, CI/CD, and relational and non-relational databases.
• Proven track record of building applications adhering to industry best practices for reliability, availability, scalability, and operability.
• Ability to manage component security analysis and collaborate with security teams.
• Strong work ethic, focus on immediate goals, and proven experience as a technical leader.
• Passion for mentoring juniors and helping them reach new heights.
• Demonstrated proficiency with programming languages, scripting languages, code quality initiatives, and operating systems.
• Excellent communication and interpersonal skills, and a strong team player.
• Gen AI expertise:
• Have experience in building and enhancing RAG pipelines.
• Have expertise in architecting, implementing and optimizing AI agents capable of planning, reasoning an acting.
• Have a working expertise in building MCP tools and clients.
• Have expertise in frameworks like Lang chain, Lang graph etc.
• Expertly craft prompts for various LLMs and contributes to fine tuning of models for specific tasks to enhance performance and alignment.
  
  

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.