Bangalore, Karnataka, India
Information Technology
Full-Time
Infogain
Overview
Roles & ResponsibilitiesKey Responsibilities
- Lead design and execution of Dataproc ? Databricks PySpark migration roadmap.
- Define modernization strategy, including data ingestion, transformation, orchestration, and governance.
- Architect scalable Delta Lake and Unity Catalog–based solutions.
- Manage and guide teams on code conversion, dependency mapping, and data validation.
- Collaborate with platform, infra, and DevOps teams to optimize compute costs and performance.
- Own the automation & GenAI acceleration layer, integrating code parsers, lineage tools, and validation utilities.
- Conduct performance benchmarking, cost optimization, and platform tuning (Photon, Auto-scaling, Delta Caching).
- Mentor senior and mid-level developers, ensuring quality standards, documentation, and delivery timelines.
- Languages: Python, PySpark, SQL
- Platforms: Databricks (Jobs, Workflows, Delta Live Tables, Unity Catalog), GCP Dataproc
- Data Tools: Hadoop, Hive, Pig, Spark (RDD & DataFrame APIs), Delta Lake
- Cloud & Integration: GCS, BigQuery, Pub/Sub, Cloud Composer, Airflow
- Automation: GenAI-powered migration tools, custom Python utilities for code conversion
- Version Control & DevOps: Git, Terraform, Jenkins, CI/CD pipelines
- Other: Performance tuning, cost optimization, and lineage tracking with Unity Catalog
- 10–14 years of data engineering experience with at least 3 years leading Databricks or Spark modernization programs.
- Proven success in migration or replatforming projects from Hadoop or Dataproc to Databricks.
- Exposure to AI/GenAI in code transformation or data engineering automation.
- Strong stakeholder management and technical leadership skills.
- 11-12 Years
- Primary Skill: Data Engineering
- Sub Skill(s): Data Engineering
- Additional Skill(s): Python, Apache Hadoop, Apache Hive, Apache Airflow, synapse, Databricks, SQL, Apache Spark, Azure Data Factory, Pyspark, GenAI Fundamentals, Cloud Pub/Sub, BigQuery
Infogain is a human-centered digital platform and software engineering company based out of Silicon Valley. We engineer business outcomes for Fortune 500 companies and digital natives in the technology, healthcare, insurance, travel, telecom, and retail & CPG industries using technologies such as cloud, microservices, automation, IoT, and artificial intelligence. We accelerate experience-led transformation in the delivery of digital platforms. Infogain is also a Microsoft (NASDAQ: MSFT) Gold Partner and Azure Expert Managed Services Provider (MSP).
Infogain, an Apax Funds portfolio company, has offices in California, Washington, Texas, the UK, the UAE, and Singapore, with delivery centers in Seattle, Houston, Austin, Kraków, Noida, Gurgaon, Mumbai, Pune, and Bengaluru.
Similar Jobs
View All
14 Hours ago
Project Manager
Finance & Banking
- 10 - 16 Yrs
- Mumbai
Hiring: Project Manager (BFSI / Capital Markets) | 10–16 Years Experience |
We are looking for an experienced Project Manager to lead and deliver technology initiatives within the BFSI and capital markets domain. This role requires strong leaders...
14 Hours ago
Senior Software Engineer II - Nest.Js
Information Technology
- 2000000 - 2200000 INR - Annual
- 4 - 7 Yrs
- Mumbai
We are seeking a Full Stack Engineer who can own systems end-to-end — from architecting backend services and infrastructure to guiding web and mobile development teams. This role is ideal for someone with deep backend expertise, strong architectural ...
15 Hours ago
SAP MRO Lead
Aerospace & Defense
- 2300000 - 2600000 INR - Annual
- 7 - 12 Yrs
- Bengaluru
Summary role description:
Hiring an SAP MRO Lead for a top global aerospace and defense innovator offering impactful, cutting-edge work.
Company description:
Our client is a leading global player in the aerospace and defense spac...
16 Hours ago
Head of Delivery
Digital Transformation & Innovation
- 15 - 20 Yrs
- Hyderabad
Roles and Responsibilties
As a Head of delivery, you are expected to have all the below skills
Manage the delivery teams working on multiple technologies across data AI/ML, LLM, integration, automation, and analytics.
Be the technol...
16 Hours ago
Digital Technology Manager - Mobile Applications
Retail & E-commerce
- 1500000 - 3500000 INR - Yearly
- 8 - 15 Yrs
- Bangalore
Position Purpose and Scope
We are looking for an experienced Mobile Application Architect who possesses strong hands-on expertise in mobile app development and architecture design. The ideal candidate will lead the end-to-end architecture, design,...
17 Hours ago
SAP Consultant
Information Technology
- 5 - 9 Yrs
- Gurgaon / Gurugram
Designation: Senior SAP Consultant
Location: Gurugram
Experience: 5–10 years
Responsibilities
End-to-end semantic layer design in Datasphere (Spaces, Data/Replication Flows, Business Builder, Analytic Models)
Building complex ana...
18 Hours ago
Backend Python Developer
Retail & E-commerce
- 8 - 12 Yrs
- Bangalore
What You’ll Work On
Build scalable backend services & APIs using Python
Design and maintain databases: MySQL, Neo4j, MongoDB
Deploy & manage containerized apps using Docker & Kubernetes (EKS)
Work with AWS services: Lambda, S3, EC2, RDS
...
19 Hours ago
Infosec Delivery Manager
Cybersecurity & Privacy
- 3000000 - 3500000 INR - Annual
- 4 - 8 Yrs
- Anywhere in India/Multiple Locations
Summary role description:
Hiring an InfoSec Delivery Manager for a rapidly scaling SaaS platform that acts as a virtual CISO for global enterprises.
Company description:
Our client is a fast-growing SaaS company that helps organi...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in