Overview
Job Summary
We are seeking a highly skilled and motivated Data Engineer to design, develop, and optimize our critical data infrastructure. You will play a pivotal role in building scalable batch and real-time data pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver impactful data solutions. If you are passionate about data, possess strong technical skills, and thrive in a collaborative environment, we encourage you to apply.
Responsibilities
We are seeking a highly skilled and motivated Data Engineer to design, develop, and optimize our critical data infrastructure. You will play a pivotal role in building scalable batch and real-time data pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver impactful data solutions. If you are passionate about data, possess strong technical skills, and thrive in a collaborative environment, we encourage you to apply.
Responsibilities
- Design, implement, and optimize robust batch and real-time data pipelines using technologies such as Spark, Python, and Airflow.
- Leverage AWS native tools (e.g., S3, EMR, Glue) effectively to build and manage scalable data solutions.
- Collaborate cross-functionally with data scientists, data analysts, product managers, and engineering teams to gather requirements, deliver high-quality data solutions, and ensure overall project success.
- Lead initiatives focused on enhancing data quality, establishing data governance frameworks, implementing robust security measures, and setting up comprehensive data monitoring systems.
- Evaluate, recommend, and integrate various cloud, big data, and DataOps tools, including SQL/NoSQL databases, streaming systems, workflow orchestration platforms, and observability tools.
- 2+ years of practical experience in data pipeline development, encompassing both batch and streaming processes.
- Advanced knowledge and hands-on experience with Spark, Python, and SQL.
- Proven expertise in designing and implementing ETL/ELT processes.
- Strong practical experience with AWS (specifically EMR, S3, Glue) or other major cloud platforms (e.g., Azure, GCP).
- Solid understanding and experience with data modeling, data warehousing concepts, and metadata management.
- Demonstrated problem-solving abilities, strong analytical skills, and a highly collaborative mindset.
- Experience with CI/CD pipelines and MLOps lifecycle.
- Exposure to NoSQL databases, data lake architectures, or streaming technologies (e.g., Kafka, Kinesis).
- Experience utilizing API tools such as Postman and Swagger.
- Awareness and understanding of data security, privacy, and compliance best practices.
Similar Jobs
View All
1 Day ago
SDE III - Frontend Heavy
Internet
- 3 - 5 Yrs
- Anywhere in India/Multiple Locations
Candidates from Saas, Software Development, Product based companies only.
About the Role:
We are looking for an experienced software engineer with strong technical and communication skills who has developed full stack web apps, created solid AP...
1 Day ago
VAYUZ Technologies - Python Developer - Django/Flask
Information Technology
Job DescriptionRole Expectations : Provide technical leadership and guidance to a team of Python developers. Mentor junior developers in best practices, coding standards, and problem-solving techniques. Participate in code reviews to ensure code ...
1 Day ago
Go (Golang) Software Engineer, Developer Tooling and Containers
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
1 Day ago
Interesting Job Opportunity: AWS Data Engineer - Python/PySpark
Information Technology
Key Responsibilities Hands on experience in data related activities such as data parsing, cleansing quality definition data pipelines, storage and ETL scripts. Expert knowledge in AWS Data Lake implementation and support (S3, Glue, DMS Athena, Lam...
1 Day ago
Go (Golang) Software Engineer, Developer Tooling and Containers
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
1 Day ago
Security Consultant (Operational Technology Security)
Information Technology
Make an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diver...
1 Day ago
Go (Golang) Software Engineer, Developer Tooling and Containers
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
1 Day ago
Senior Software Engineer - Digital Workplace
Information Technology
Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is very widely used in breakthrough enterprise initiatives such as public cloud, data science, AI, e...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in