Bangalore, Karnataka, India
Information Technology
Full-Time
Josys
Overview
Location: Bengaluru, Karnataka, India
About the Role:
We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast.
What You'll Do:
About the Role:
We're looking for an experienced Senior Data Engineer (6-8 years) to join our data team. You'll be key in building and maintaining our data systems on AWS. You'll use your strong skills in big data tools and cloud technology to help our analytics team get valuable insights from our data. You'll be in charge of the whole process of our data pipelines, making sure the data is good, reliable, and fast.
What You'll Do:
- Design and build efficient data pipelines using Spark / PySpark / Scala.
- Manage complex data processes with Airflow, creating and fixing any issues with the workflows (DAGs).
- Clean, transform, and prepare data for analysis.
- Use Python for data tasks, automation, and building tools.
- Work with AWS services like S3, Redshift, EMR, Glue, and Athena to manage our data infrastructure.
- Collaborate closely with the Analytics team to understand what data they need and provide solutions.
- Help develop and maintain our Node.js backend, using Typescript, for data services.
- Use YAML to manage the settings for our data tools.
- Set up and manage automated deployment processes (CI/CD) using GitHub Actions.
- Monitor and fix problems in our data pipelines to keep them running smoothly.
- Implement checks to ensure our data is accurate and consistent.
- Help design and build data warehouses and data lakes.
- Use SQL extensively to query and work with data in different systems.
- Work with streaming data using technologies like Kafka for real-time data processing.
- Stay updated on the latest data engineering technologies.
- Guide and mentor junior data engineers.
- Help create data management rules and procedures.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 6-8 years of experience as a Data Engineer.
- Strong skills in Spark and Scala for handling large amounts of data.
- Good experience with Airflow for managing data workflows and understanding DAGs.
- Solid understanding of how to transform and prepare data.
- Strong programming skills in Python for data tasks and automation..
- Proven experience working with AWS cloud services (S3, Redshift, EMR, Glue, IAM, EC2, and Athena).
- Experience building data solutions for Analytics teams.
- Familiarity with Node.js for backend development.
- Experience with Typescript for backend development is a plus.
- Experience using YAML for configuration management.
- Hands-on experience with GitHub Actions for automated deployment (CI/CD).
- Good understanding of data warehousing concepts.
- Strong database skills - OLAP/OLTP
- Excellent command of SQL for data querying and manipulation.
- Experience with stream processing using Kafka or similar technologies.
- Excellent problem-solving, analytical, and communication skills.
- Ability to work well independently and as part of a team.
- Familiarity with data lake technologies (e.g., Delta Lake, Apache Iceberg).
- Experience with other stream processing technologies (e.g., Flink, Kinesis).
- Knowledge of data management, data quality, statistics and data governance frameworks.
- Experience with tools for managing infrastructure as code (e.g., Terraform).
- Familiarity with container technologies (e.g., Docker, Kubernetes).
- Experience with monitoring and logging tools (e.g., Prometheus, Grafana).
Similar Jobs
View All
3 Hours ago
Data Analyst (Kannada Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Karnataka, India
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
9 Hours ago
Data Analyst (Marathi Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- India, Pune
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data...
9 Hours ago
Data Analyst (Telugu Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Andhra Pradesh, Telangana, India
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
1 Day ago
Java Developer – Payments Domain
Information Technology
- 4 - 7 Yrs
- Mumbai (All Areas)
We are hiring Java Developers with 4–6 years of hands-on experience in backend development, particularly within the Payments or FinTech domain. The ideal candidate should possess a strong foundation in Java technologies and be capable of working in a...
1 Day ago
SAP Functional Architect
Information Technology
- 40,00,000 - 45,00,000 INR - Annual
- 12 - 15 Yrs
- Bangalore, Chennai
We are seeking an experienced SAP Pre-Sales Architect with a strong functional background and deep expertise in at least one SAP functional area. The ideal candidate will have extensive knowledge of cross-module integrations and a proven track record...
1 Day ago
Senior React Native Developer
Information Technology
- 7 - 12 Yrs
- Jaipur
The NineHertz is on the lookout for a Senior React Native Developer who is passionate about mobile app development and thrives in a fast-paced environment. This is a fantastic opportunity to work with a dynamic team, drive innovation, and help delive...
1 Day ago
Senior Data & AI Analytics Engineer (Remote)
AI & Machine Learning Advancement
- 18,00,000 - 24,00,000 INR - Annual
- 5 - 8 Yrs
- Pune
Job Ref: NT-DAAI-003
Experience: 5–8 years
Client: A prestigious AI-first tech company
Notice: Early joiners preferred (Immediate- 30 days)
We are hiring on behalf of a prestigious AI-first technology client for a Senior Data & AI Analytics En...
1 Day ago
AI Engineering Manager (Remote)
Information Technology
- 40,00,000 - 50,00,000 INR - Annual
- 10 - 15 Yrs
- Pune
Experience: 10 to 15 years
Location: Remote
Notice Period: Immediate to 30 days preferred
Client: Leading mid-sized firm specializing in AI-driven solutions
Overview:
We are looking for an AI Engineering Manager to lead a dynamic team of ...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in