Pune, Maharashtra, India
Information Technology
Contract
Josys
Overview
Senior Data Engineer (Contract)
Location: Bengaluru, Karnataka, India
About The Role
We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast.
What You'll Do
Location: Bengaluru, Karnataka, India
About The Role
We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast.
What You'll Do
- Design and build efficient data pipelines using Spark / PySpark / Scala.
- Manage complex data processes with Airflow, creating and fixing any issues with the workflows (DAGs).
- Clean, transform, and prepare data for analysis.
- Use Python for data tasks, automation, and building tools.
- Work with AWS services like S3, Redshift, EMR, Glue, and Athena to manage our data infrastructure.
- Collaborate closely with the Analytics team to understand what data they need and provide solutions.
- Help develop and maintain our Node.js backend, using Typescript, for data services.
- Use YAML to manage the settings for our data tools.
- Set up and manage automated deployment processes (CI/CD) using GitHub Actions.
- Monitor and fix problems in our data pipelines to keep them running smoothly.
- Implement checks to ensure our data is accurate and consistent.
- Help design and build data warehouses and data lakes.
- Use SQL extensively to query and work with data in different systems.
- Work with streaming data using technologies like Kafka for real-time data processing.
- Stay updated on the latest data engineering technologies.
- Guide and mentor junior data engineers.
- Help create data management rules and procedures.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 6-8 years of experience as a Data Engineer.
- Strong skills in Spark and Scala for handling large amounts of data.
- Good experience with Airflow for managing data workflows and understanding DAGs.
- Solid understanding of how to transform and prepare data.
- Strong programming skills in Python for data tasks and automation..
- Proven experience working with AWS cloud services (S3, Redshift, EMR, Glue, IAM, EC2, and Athena).
- Experience building data solutions for Analytics teams.
- Familiarity with Node.js for backend development.
- Experience with Typescript for backend development is a plus.
- Experience using YAML for configuration management.
- Hands-on experience with GitHub Actions for automated deployment (CI/CD).
- Good understanding of data warehousing concepts.
- Strong database skills - OLAP/OLTP
- Excellent command of SQL for data querying and manipulation.
- Experience with stream processing using Kafka or similar technologies.
- Excellent problem-solving, analytical, and communication skills.
- Ability to work well independently and as part of a team.
- Familiarity with data lake technologies (e.g., Delta Lake, Apache Iceberg).
- Experience with other stream processing technologies (e.g., Flink, Kinesis).
- Knowledge of data management, data quality, statistics and data governance frameworks.
- Experience with tools for managing infrastructure as code (e.g., Terraform).
- Familiarity with container technologies (e.g., Docker, Kubernetes).
- Experience with monitoring and logging tools (e.g., Prometheus, Grafana).
Similar Jobs
View All
13 Hours ago
Senior Data Engineer(Product Companies Only)
Internet
- 4 - 8 Yrs
- Anywhere in India/Multiple Locations
My Client is an AI powered, all-in-one white-label sales & marketing platform that empowers agencies, entrepreneurs, and businesses to elevate their digital presence and drive growth.
Role : Senior Data Engineer
Location : Remote
They are se...
16 Hours ago
SDE III FullStack(Backend Heavy)
Information Technology
- 3 - 8 Yrs
- Anywhere in India/Multiple Locations
As an SDE3, you’ll work on backend-heavy (70% Backend Node.Js & 30% Frontend React.Js)full stack features that bring real-world booking experiences to life — such as workflows, payments, lead generation, form submissions, notifications, and search....
1 Day ago
Project Manager SAP
Information Technology
- 30 - 35 INR - Annual
- 10 - 15 Yrs
- Pune
🚀 We’re Hiring – SAP Project Manager | Pune 🚀
Exciting opportunity with my client in Pune for an experienced SAP Project Manager to lead end-to-end S/4HANA implementations (Rise & GROW) and AMS projects in a dynamic, global environment.
We’re l...
2 Days ago
Android Developer - Mobile App Integration
Information Technology
- Pune, Maharashtra, India
Responsibilities Design and develop advanced applications for the Android platform Collaborate with cross-functional teams to define, design, and ship new features Identify and fix bugs and performance bottlenecks Continuously discover, evaluate...
2 Days ago
Full Stack Developer in Delhi
Information Technology
- Pune, Maharashtra, India
Key Responsibilities Architect, develop, and maintain high-performance, scalable web applications capable of handling rapid user growth. Design and implement distributed, fault-tolerant systems with load balancing, caching, and database sharding a...
2 Days ago
Lead Data Scientist
Information Technology
- Pune, Maharashtra, India
Our PurposeMastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments ch...
2 Days ago
Software Engineer (Wireless & Smart Spaces) - C/C++, Python, TCP/IP, Wireless Networking, 4+ years of experience
Information Technology
- Pune, Maharashtra, India
Join Cisco Wireless and help transform the network experience with our innovative wireless solutions. Enhance speed and security, and apply AI/ML for real-time insights, continuous monitoring and seamless Wi-Fi connectivity including the latest Wi-F...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in