Thiruvananthapuram, Kerala, India
Information Technology
Full-Time
Citi
Overview
Responsible for designing, developing, and optimizing data processing solutions using a combination of Big Data technologies. Focus on building scalable and efficient data pipelines for handling large datasets and enabling batch & real-time data streaming and processing.
Responsibilities:
Responsibilities:
- > Develop Spark applications using Scala or Python (Pyspark) for data transformation, aggregation, and analysis.
> Develop and maintain Kafka-based data pipelines: This includes designing Kafka Streams, setting up Kafka Clusters, and ensuring efficient data flow.
> Create and optimize Spark applications using Scala and PySpark: They leverage these languages to process large datasets and implement data transformations and aggregations.
> Integrate Kafka with Spark for real-time processing: They build systems that ingest real-time data from Kafka and process it using Spark Streaming or Structured Streaming.
> Collaborate with data teams: This includes data engineers, data scientists, and DevOps, to design and implement data solutions.
> Tune and optimize Spark and Kafka clusters: Ensuring high performance, scalability, and efficiency of data processing workflows.
> Write clean, functional, and optimized code: Adhering to coding standards and best practices.
> Troubleshoot and resolve issues: Identifying and addressing any problems related to Kafka and Spark applications.
> Maintain documentation: Creating and maintaining documentation for Kafka configurations, Spark jobs, and other processes.
> Stay updated on technology trends: Continuously learning and applying new advancements in functional programming, big data, and related technologies.
Proficiency in:
Hadoop ecosystem big data tech stack(HDFS, YARN, MapReduce, Hive, Impala).
Spark (Scala, Python) for data processing and analysis.
Kafka for real-time data ingestion and processing.
ETL processes and data ingestion tools
Deep hands-on expertise in Pyspark, Scala, Kafka
Programming Languages:
Scala, Python, or Java for developing Spark applications.
SQL for data querying and analysis.
Other Skills:
Data warehousing concepts.
Linux/Unix operating systems.
Problem-solving and analytical skills.
Version control systems
------------------------------------------------------
Job Family Group:
Technology
------------------------------------------------------
Job Family:
Applications Development
------------------------------------------------------
Time Type:
Full time
------------------------------------------------------
Most Relevant Skills
Please see the requirements listed above.
------------------------------------------------------
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.
------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.
Similar Jobs
View All
1 Day ago
Software Developer 4
Information Technology
- Bangalore, Karnataka, India
Job DescriptionDesign, develop, troubleshoot and debug software programs for databases, applications, tools, networks etc.ResponsibilitiesAs a member of the software engineering division, you will take an active role in the definition and evolution ...
1 Day ago
Software Developer (ReactJS & DotNet) in Faridabad, Noida, Palwal, Ballabgarh
Information Technology
- Bangalore, Karnataka, India
As a Software Developer at Digital Shahi Pvt. Ltd., you will have the opportunity to work on cutting-edge technologies such as ReactJS, DotNet, MongoDB, MERN stack, and Node.js. Join our dynamic team and be a part of revolutionizing the digital land...
1 Day ago
iOS Developer - Objective C/SWIFT
Information Technology
- Bangalore, Karnataka, India
Job DescriptionWe are looking for a skilled and enthusiastic iOS Developer to join our team. The candidate will have experience developing and maintaining mobile applications on the iOS platform and possess a passion for creating user-friendly and h...
1 Day ago
Manual/Automation Tester
Information Technology
- Thiruvananthapuram, Kerala, India
Job DescriptionThis is a full-time on-site role located in Hyderabad. As a Testing professional with 0-2 year of experience, you will be responsible for conducting testing activities, including developing and executing test plans, test cases, and te...
1 Day ago
Sr. Software Engineer - iOS Development
Information Technology
- Thiruvananthapuram, Kerala, India
Position Description: As a Senior Software Engineer, iOS Developer, you will contribute to the development of intuitive, high-performance iOS applications for enterprise use. This role focuses on implementing features and improving the product exper...
1 Day ago
Sr. Software Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
What you will do: Design, develop, enhance, and test software / embedded software for medical and connectivity products for R&D department. Closely work with peers in software engineering, software quality assurance and project & product management ...
1 Day ago
PHP Developer - Laravel Frameworks
Information Technology
- Thiruvananthapuram, Kerala, India
Location: Pune, Bengaluru, Chennai, Hyderabad, GurugramWork Model: Hybrid (3 days from office)Experience Required: 10+ yearsRole Type: Individual ContributorClient: US-based multinational banking institutionNotice Period: Immediate to 21 daysRole Su...
1 Day ago
Data Engineer - Generative AI
Information Technology
- Thiruvananthapuram, Kerala, India
About The RoleWe are seeking a skilled and passionate Data Engineer to join our team and drive the development of scalable data pipelines for Generative AI (GenAI) and Large Language Model (LLM)-powered applications. This role demands hands-on exper...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in