Bangalore, Karnataka, India
Information Technology
Full-Time
Citi
Overview
Responsible for designing, developing, and optimizing data processing solutions using a combination of Big Data technologies. Focus on building scalable and efficient data pipelines for handling large datasets and enabling batch & real-time data streaming and processing.
Responsibilities:
Responsibilities:
- > Develop Spark applications using Scala or Python (Pyspark) for data transformation, aggregation, and analysis.
> Develop and maintain Kafka-based data pipelines: This includes designing Kafka Streams, setting up Kafka Clusters, and ensuring efficient data flow.
> Create and optimize Spark applications using Scala and PySpark: They leverage these languages to process large datasets and implement data transformations and aggregations.
> Integrate Kafka with Spark for real-time processing: They build systems that ingest real-time data from Kafka and process it using Spark Streaming or Structured Streaming.
> Collaborate with data teams: This includes data engineers, data scientists, and DevOps, to design and implement data solutions.
> Tune and optimize Spark and Kafka clusters: Ensuring high performance, scalability, and efficiency of data processing workflows.
> Write clean, functional, and optimized code: Adhering to coding standards and best practices.
> Troubleshoot and resolve issues: Identifying and addressing any problems related to Kafka and Spark applications.
> Maintain documentation: Creating and maintaining documentation for Kafka configurations, Spark jobs, and other processes.
> Stay updated on technology trends: Continuously learning and applying new advancements in functional programming, big data, and related technologies.
Proficiency in:
Hadoop ecosystem big data tech stack(HDFS, YARN, MapReduce, Hive, Impala).
Spark (Scala, Python) for data processing and analysis.
Kafka for real-time data ingestion and processing.
ETL processes and data ingestion tools
Deep hands-on expertise in Pyspark, Scala, Kafka
Programming Languages:
Scala, Python, or Java for developing Spark applications.
SQL for data querying and analysis.
Other Skills:
Data warehousing concepts.
Linux/Unix operating systems.
Problem-solving and analytical skills.
Version control systems
------------------------------------------------------
Job Family Group:
Technology
------------------------------------------------------
Job Family:
Applications Development
------------------------------------------------------
Time Type:
Full time
------------------------------------------------------
Most Relevant Skills
Please see the requirements listed above.
------------------------------------------------------
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.
------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.
Similar Jobs
View All
1 Hour ago
Product Designer
Fintech
- 1800000 - 2500000 INR - Annual
- 3 - 8 Yrs
- Mumbai, Navi Mumbai, Thane
Job Description :-
Looking for potential candidates only from Product Based Organization who have implemented B2C projects.
● A Compelling Portfolio:
A link to your online portfolio is required for consideration. Your portfolio must showc...
5 Hours ago
Junior Android Developer
Information Technology
- 1000000 - 1200000 INR - Annual
- 1 - 3 Yrs
- Pune
About the Client:
A leading global provider of in-flight entertainment and connectivity (IFEC) solutions, serving major airlines worldwide with advanced hardware, software, and communication systems.
About the Role:
Title: MTS I Android
Lo...
1 Day ago
Software Engineer III-Salesforce
Information Technology
- Bangalore, Karnataka, India
About the job:Key responsibilities: 1. Formal training or certification in Salesforce concepts with 3+ years of hands-on experience in Salesforce customization. 2. Practical experience in system design, application development, testing, and maintai...
1 Day ago
Lead Software Engineer - Python, AWS, Big Data
Information Technology
- Bangalore, Karnataka, India
About the job:Key responsibilities: 1. Build and deploy data processing pipelines and applications in production environments. 2. Execute creative software design, development, and troubleshooting while thinking beyond conventional approaches to so...
1 Day ago
Full Stack Developer in Delhi, Gaya, Patna, Dehri, Sasaram, Mumbai (Hybrid)
Information Technology
- Bangalore, Karnataka, India
Key Responsibilities Build user interfaces (UI) with HTML, CSS, JavaScript Work with frameworks like React / Vue / Angular Ensure responsive and cross-browser compatible designs Develop APIs and server-side logic Work with backend frameworks (L...
1 Day ago
Scrum Master, AVP
Information Technology
- Bangalore, Karnataka, India
Join us as a Scrum Master We’re looking for an experienced Scrum Master to drive a culture of continuous improvement You’ll make sure that your team lives by the values of Scrum, empowering them to become high performing to increase the productivity,...
1 Day ago
Data Scientist
Information Technology
- Bangalore, Karnataka, India
Job Requirements At Quest Global, it’s not just what we do but how and why we do it that makes us different. With over 25 years as an engineering services provider, we believe in the power of doing things differently to make the impossible possible....
1 Day ago
Scrum Master
Information Technology
- Bangalore, Karnataka, India
At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive tr...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in