Free cookie consent management tool by TermsFeed Data Engineer - Sr. Consultant-Java/SQL/Spark (12+yrs) | Antal Tech Jobs
Back to Jobs
15 Weeks ago

Data Engineer - Sr. Consultant-Java/SQL/Spark (12+yrs)

decor
Vadodara, Gujarat, India
Information Technology
Full-Time
Visa

Overview

Company Description


Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.


Job Description


The Universal Data Catalog (UDC) / Visa Data Catalog (VDC) is a comprehensive metadata management platform designed to provide a centralized repository for all data-related information across the organization. The platform enables efficient data discovery, governance, and utilization by offering detailed metadata definitions for tables, columns, and other data assets. It supports various business units by improving data accessibility, enhancing data quality, and facilitating compliance with data governance policies. Our implementation leverages the open-source Datahub project, ensuring a robust, flexible, and scalable solution.

Key Responsibilities

  • Develop and maintain the Enterprise Data Catalog / Visa Data Catalog platform, ensuring it meets the organization’s evolving needs.
  • Implement and manage metadata ingestion processes to ensure the catalog is up-to-date with the latest data definitions and business context.
  • Collaborate with data stewards, data owners, and other stakeholders to enrich metadata with business definitions, data lineage, and usage context.
  • Enhance the catalog’s search and discovery capabilities to provide users with intuitive and efficient access to data assets.
  • Integrate the data catalog with other enterprise systems and tools to support data governance, data quality, and analytics initiatives.
  • Monitor and optimize the performance of the data catalog to ensure scalability and reliability.
  • Provide training and support to users on how to effectively leverage the data catalog for their data-related needs.
  • Actively collaborate with the open-source community to contribute to and leverage the latest advancements in the Datahub project.
  • Analyze industry best practices and keep the catalog functionality up-to-date with feature sets provided by the market, while focusing on Visa’s scalability requirements.
  • Champion the adoption of open infrastructure solutions that are fit for purpose while keeping technology relevant.
  • Spend 80% of the time writing code in different languages, frameworks, and technology stacks.

This is a hybrid position. Expectation of days in office will be confirmed by your Hiring Manager.


Qualifications

Basic Qualifications
8+ years of relevant work experience with a Bachelor’s Degree or at least 5 years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 2 years of work experience with a PhD, OR 11+ years of relevant work experience.

Preferred Qualifications
9 or more years of relevant work experience with a Bachelor Degree or 7 or more relevant years of experience with an Advanced Degree (e.g. Masters, MBA, JD, MD) or 3 or more years of experience with a PhD
  • 10 yrs+ of experience in architecture design and development of large-scale data management platforms and data application with simple solutions
  • Must have extensive hands-on coding and designing skills on
oJava/Python or Node.js/Javascript for backend
oMVC (model-view-controller) for end-to-end development
oSQL/NoSQL technology. Familiar with Databases like Oracle, DB2, SQL Server, etc.
oWeb Services (REST/ SOAP/gRPC)
oReact/Angular for front-end (UI front-end nice to have)
  • Expertise in design and management of complex data structures and data processes
  • Expertise in efficiently leveraging the power of distributed big data systems, including but not limited to Hadoop Hive, Spark, Kafka streaming, etc.
  • Deep knowledge and hands on experience on big data and cloud computing technologies.
  • Strong service architecture and development experience with high performance and scalability
  • Technical background in data with deep understanding of issues in multiple areas such as data acquisition, ingestion and processing, data management, distributed processing, and high availability is required.
  • Strong on driving for results and self-motivated, strong learning mindset, with good understanding of related advanced/new technology. Keep up with the technology development in the related areas in the industry, which could be leveraged to enhance current architectures and build durable new ones.
  • Bachelor’s degree in computer science or related technical discipline required. Advanced degree is a plus.
  • Strong leadership and team player.
  • Strong skills on mentoring/growing junior people
  • Payment industry experience is a plus.

What will also help:
  • Experience with ETL/ELT tools / applications
  • Experience with Apache NiFi and Apache Spark for processing large data sets
  • Experience on Elastic Search
  • Knowledge on Data Catalog tools
  • Experience in building Data Pipeline development tools
  • Experience with Data Governance and Data Quality tools

Additional Information


Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Share job
Similar Jobs
View All
20 Hours ago
Data Analyst
Information Technology
  • 300000 - 1200000 INR - Annual
  • 1 - 7 Yrs
  • Anywhere in India/Multiple Locations
Location: Remote – India Experience: 1–7 years Employment Type: Full-time & Part-time options available About the Role We’re seeking analytical minds who love turning data into actionable insights. As a Data Analyst, you’ll work closely w...
decor
20 Hours ago
Sr. ServiceNow Developer | 8+ Years | Pune
Information Technology
  • 30 - 40 INR - Annual
  • 8 - 11 Yrs
  • Pune
About the Role: This is a senior-level ServiceNow Developer role where you’ll be responsible for driving the technical vision and execution of complex ServiceNow solutions across the organization. What We’re Looking For: Strong hands-...
decor
20 Hours ago
Mobile Developer
Information Technology
  • 350000 - 1000000 INR - Annual
  • 1 - 6 Yrs
  • Anywhere in India/Multiple Locations
About the Role We're looking for talented Mobile Developers—whether you're a rising star with a year of experience or a seasoned engineer with eight—to build intuitive, high-performance mobile applications for Android and iOS. You’ll collaborate w...
decor
2 Days ago
IT Support Executive
Information Technology
  • Hyderabad, Telangana, India
hackajob is collaborating with British Airways to connect them with exceptional tech professionals for this role.A career without limitsAs the nation’s flag carrier, we take great pride in connecting Britain with the world and the world with Britain...
decor
2 Days ago
Full Stack Developer (Java + AWS DevOps Focus)
Information Technology
  • Hyderabad, Telangana, India
Company Description We are a Stealth GenAI B2B Startup focused on revolutionizing construction industry using Artificial General Intelligence. Our team is comprised of visionaries, researchers, and engineers dedicated to pushing the boundaries of AI...
decor
2 Days ago
DevOps Engineer
Information Technology
  • Surat, Gujarat, India
About QuantaniteQuantanite is a customer experience (CX) solutions company that helps fast-growing companies and leading global brands to transform and grow. We do this through a collaborative and consultative approach, rethinking business processes...
decor
2 Days ago
Tech Carrot - Data Engineer - Python/ETL
Information Technology
  • Hyderabad, Telangana, India
Skills Advanced working knowledge and experience with relational and non-relational databases. Experience building and optimizing Big Data pipelines, architectures, and datasets. Strong analytic skills related to working with structured and unstr...
decor
2 Days ago
Software Engineer - Java8 & React Fullstack_C2E
Information Technology
  • Surat, Gujarat, India
Reference 250008EIResponsibilitiesJob description:As a full stack developer, you’ll will work in agile. You will participate to all the agile ceremonies (sprint planning, refinement, retrospective, poker planning et sprint review)As part of the Pari...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media