Pune, Maharashtra, India
Finance & Banking
Full-Time
UST
Overview
Role Description
Must-Have Skills:
GCP Services, ,Dataproc,DataFlow
Must-Have Skills:
- 5+ years of experience as a Big Data Engineer
- 3+ years of experience with Apache Spark, Hive, HDFS, and Beam (optional)
- Strong proficiency in SQL and either Scala or Python
- Experience with ETL processes and working with structured and unstructured data
- 2+ years of experience with Cloud Platforms (GCP, AWS, or Azure)
- Hands-on experience with software build management tools like Maven or Gradle
- Experience in automation, performance tuning, and optimizing data pipelines
- Familiarity with CI/CD, serverless computing, and infrastructure-as-code practices
- Experience with Google Cloud Services (BigQuery, Dataproc, Dataflow, Composer, DataStream)
- Strong knowledge of data pipeline development and optimization
- Familiarity with source control tools (SVN/Git, GitHub)
- Experience working in Agile environments (Scrum, XP, etc.)
- Knowledge of relational databases (SQL Server, Oracle, MySQL)
- Experience with Atlassian tools (JIRA, Confluence, GitHub)
- Extract, transform, and load (ETL) data from multiple sources using Big Data technologies
- Develop, enhance, and support data ingestion jobs using GCP services like Apache Spark, Dataproc, Dataflow, BigQuery, and Airflow
- Work closely with senior engineers and cross-functional teams to improve data accessibility
- Automate manual processes, optimize data pipelines, and enhance infrastructure for scalability
- Modify data extraction pipelines to follow standardized, reusable approaches
- Optimize query performance and data access techniques in collaboration with senior engineers
- Follow modern software development practices, including microservices, CI/CD, and infrastructure-as-code
- Participate in Agile development teams, ensuring best practices for software engineering and data management
- Bachelor's degree in Computer Science, Systems Engineering, or a related field
- Self-starter with strong problem-solving skills and adaptability to shifting priorities
- Cloud certifications (GCP, AWS, or Azure) are a plus
GCP Services, ,Dataproc,DataFlow
Similar Jobs
View All
1 Day ago
TrueFan - Senior Machine Learning Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
About UsTrueFan is at the forefront of AI-driven content generation, leveraging cutting-edge generative models to build next-generation products. Our mission is to redefine content generation space through advanced AI technologies, including deep ge...
1 Day ago
Salesforce commerce cloud consultant
Information Technology
- Thiruvananthapuram, Kerala, India
Salesforce Commerce Cloud consultant 5+ Years of Experience 6 to 12 months Mode - Remote 1.1LPM - 1.2LPM Max Key Responsibilities Translate business requirements into scalable Salesforce Service Cloud solutions, in collaboration with CAE's technic...
1 Day ago
Cloud Infrastructure Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
DescriptionInvent the future with us. Recognized by Fast Company’s 2023 100 Best Workplaces for Innovators List, Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focuse...
1 Day ago
Devops Engineer- Intermetiate
Information Technology
- Thiruvananthapuram, Kerala, India
BackJD: Dev ops Engineer:As a DevOps Specialist- should be able to take ownership of the entire DevOps process, including Automated CI/CD pipelines and deployment to production.They should also be comfortable with risk analysis and prioritization.Le...
1 Day ago
Sr Data Scientist (London)
Information Technology
- Thiruvananthapuram, Kerala, India
AryaXAI stands at the forefront of AI innovation, revolutionizing AI for mission-critical, highly regulated industries by building explainable, safe, and aligned systems that scale responsibly. Our mission is to create AI tools that empower research...
1 Day ago
Software Test Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further att...
1 Day ago
Software Developer 5 (Java Fullstack)
Information Technology
- Thiruvananthapuram, Kerala, India
Job DescriptionBuilding off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team focuses on product development and product strategy for Oracle Health, while building out a complete platfo...
1 Day ago
Java Developer - Spring Frameworks
Information Technology
- Thiruvananthapuram, Kerala, India
Java DescriptionWe are looking for a passionate and talented Java Developer with 2-3 years of hands-on experience to join our growing development team.The ideal candidate should have a strong foundation in Java technologies and the ability to develo...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in