Bangalore, Karnataka, India
Information Technology
Full-Time
UST
Overview
Role Description
Roles & Responsibilities:
Development & Implementation
Spark,Hadoop,Hive,Gcp
Roles & Responsibilities:
Development & Implementation
- Design, build, and maintain large-scale batch and real-time data pipelines using PySpark, Spark, Hive, and related big data tools.
- Write clean, efficient, and scalable code aligned with application design and coding standards.
- Create and maintain technical documentation including design documents, test cases, and configurations.
- Contribute to HLD, LLD, and data architecture documents.
- Review and validate designs and code from peers and junior developers.
- Lead technical discussions and decisions with cross-functional teams.
- Optimize data processing workflows for efficiency, cost, and performance.
- Manage data quality and ensure data accuracy, lineage, and governance across the pipeline.
- Collaborate with product managers, data stewards, and business stakeholders to translate data requirements into robust engineering solutions.
- Clarify requirements and propose design options to customers.
- Write and review unit tests and integration tests to ensure data integrity and performance.
- Monitor and troubleshoot data pipeline issues and ensure minimal downtime.
- Participate in sprint planning, estimation, and daily stand-ups.
- Ensure on-time delivery of user stories and bug fixes.
- Drive release planning and execution processes.
- Set FAST goals and provide timely feedback to team members.
- Mentor junior engineers, contribute to a positive team environment, and drive continuous improvement.
- Ensure adherence to compliance standards such as SOX, HIPAA, and organizational coding standards.
- Contribute to knowledge repositories, project wikis, and best practice documents.
- Minimum 6+ years of experience as a Data Engineer.
- Hands-on expertise in PySpark and SQL.
- Experience in Google Cloud Platform (GCP) or similar cloud environments (AWS, Azure).
- Proficient in Big Data technologies such as Spark, Hadoop, Hive.
- Solid understanding of ETL/ELT frameworks, data warehousing, and data modeling.
- Strong knowledge of CI/CD tools (Jenkins, Git, Ansible, etc.).
- Excellent problem-solving and analytical skills.
- Strong written and verbal communication skills.
- Experience with Agile/Scrum methodologies.
- Experience with data orchestration tools (Airflow, Control-M).
- Familiarity with modern data platforms such as Snowflake, DataRobot, Denodo.
- Experience in containerized environments (Kubernetes, Docker).
- Exposure to data security, governance, and compliance frameworks.
- Hands-on with Terraform, ARM Templates, or similar scripting tools for infrastructure automation.
- Domain knowledge in banking, healthcare, or retail industries.
Spark,Hadoop,Hive,Gcp
Similar Jobs
View All
1 Day ago
Java Full Stack Developer
Information Technology
- Bangalore, Karnataka, India
Primary Skill - Java, Springboot, Angular, React.js Responsibilities: Lead and mentor a team of Java & Springboot Developers in the design, development, and maintenance of applications. Work with business stakeholders and technical teams to gather...
1 Day ago
Python Developer, Analyst, Firm Risk Management
Information Technology
- Bangalore, Karnataka, India
Morgan StanleyPython Developer - AnalystProfile DescriptionWe’re seeking someone to join our team as (Analyst - Python Developer) in (Credit Risk) division.Firm Risk ManagementIn the Firm Risk Management division, we advise businesses across the Fir...
1 Day ago
Junior SQL Developer/Software Integration Specialist
Information Technology
- Bangalore, Karnataka, India
Position : Junior SQL Developer / Software Integration SpecialistLocation : MumbaiExperience : Fresher or 1 Year of Relevant ExperienceJob Type : Full-timeDepartment : IT / DevelopmentKey skills : Sybase Power Builder/PB 6.5 /POWER BUILDERJob Overvi...
1 Day ago
Agile Coach
Information Technology
- Bangalore, Karnataka, India
Job PurposeResponsible to ensure a successful Agile adoption within the organization and thereby within the scrum teams and within the different organization disciplines. Lead the way to better deliver done-high-quality products, through continuous ...
1 Day ago
Python Developer - Data Platform
Information Technology
- Bangalore, Karnataka, India
Key Responsibilities Develop, maintain, and optimize robust data ingestion and ETL/ELT pipelines using Python on the Databricks platform. Work collaboratively with the architecture team to align implementation with long-term data strategies. Leve...
1 Day ago
AWS Admin And AWS Architect
Information Technology
- Bangalore, Karnataka, India
Primary skills:Technology->IOT Platform->AWS IOT A day in the life of an Infoscion As part of the Infosys consulting team, your primary role would be to get to the heart of customer issues, diagnose problem areas, design innovative solutions and fa...
1 Day ago
Machine Learning Engineer - Data Modeling
Information Technology
- Bangalore, Karnataka, India
Role : Machine Learning and AI EngineerExp : 4+ YearsType : HybridLocation : Bangalore We are looking for a skilled Mid-Level AI Engineer with extensive experience in building full-stack AI solutions. The ideal candidate will have a strong foundat...
1 Day ago
Interesting Job Opportunity: Target Integration - Principal Software Engineer - Node.js/Python
Information Technology
- Chennai, Tamil Nadu, India
Key Responsibilities Drive the end-to-end architecture, design, and implementation of robust, scalable, and secure software systems. Define technical standards, frameworks, and best practices, ensuring delivery excellence. Collaborate with stakeh...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in