Coimbatore, Tamil Nadu, India
Social Good & Community Development, Information Technology
Full-Time
Oblige IT
Overview
Key Responsibilities
- Design, develop, and optimize large-scale data pipelines using PySpark and Apache Spark.
- Build scalable and robust ETL workflows leveraging AWS services such as EMR, S3, Lambda, and Glue.
- Collaborate with data scientists, analysts, and other engineers to gather requirements and deliver clean, well-structured data solutions.
- Integrate data from various sources, ensuring high data quality, consistency, and reliability.
- Manage and schedule workflows using Apache Airflow.
- Work on ML model deployment pipelines using tools like SageMaker and Anaconda.
- Write efficient and optimized SQL queries for data processing and validation.
- Develop and maintain technical documentation for data pipelines and architecture.
- Participate in Agile ceremonies, sprint planning, and code reviews.
- Troubleshoot and resolve issues in production environments with minimal supervision.
- Bachelor's or Masters degree in Computer Science, Engineering, or a related field.
- Python
- PySpark
- SQL
- AWS (EMR, EC2, S3, Lambda, Glue)
- Experience in developing and orchestrating pipelines using Apache Airflow.
- Familiarity with SageMaker for ML deployment and Anaconda for environment management.
- Proficiency in working with large datasets and optimizing Spark jobs.
- Experience in building data lakes and data warehouses on AWS.
- Strong understanding of data governance, data quality, and data lineage.
- Excellent documentation and communication skills.
- Comfortable working in a fast-paced Agile environment.
- Experience with Kafka or other real-time streaming platforms.
- Familiarity with DevOps practices and tools (e.g., Terraform, CloudFormation).
- Exposure to NoSQL databases such as DynamoDB or MongoDB.
- Knowledge of data security and compliance standards (GDPR, HIPAA).
- Work with cutting-edge technologies in a collaborative and innovative environment.
- Opportunity to influence large-scale data infrastructure.
- Competitive salary, benefits, and professional development support.
- Be part of a growing team solving real-world data challenges.
Similar Jobs
View All
16 Hours ago
Data Analyst (Odia Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Jharkhand, Andhra Pradesh, Odisha
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
16 Hours ago
Data Analyst (Kannada Speakers)
AI & Machine Learning Advancement
- 1 - 20 Yrs
- Karnataka, India
For thousands of years, maps have provided humans with the knowledge they need to make decisions. As a Maps Evaluator, you will have the opportunity to provide ground truth for your town, city or country.
At Peroptyx, we are looking for Data Ana...
18 Hours ago
Technical Lead - Backend Development - Node.Js
Finance & Banking
- 50,00,000 - 55,00,000 INR - Annual
- 6 - 8 Yrs
- Bangalore
What youʼll be doing
We are much more than our job descriptions, but here is where you will begin:
● Collaborate with stakeholders, including product owners, project managers, and scrum masters, to define and clarify project requirements.
● Transl...
19 Hours ago
Engineering Manager
Finance & Banking
- 55,00,000 - 60,00,000 INR - Annual
- 8 - 12 Yrs
- Bangalore
What youʼll be doing
Weʼre much more than our job descriptions, but hereʼs where youʼll begin:
● Lead and deliver large-scale platform and product initiatives that impact millions of users.
● Collaborate with product, design, and business teams to...
23 Hours ago
Technical Project Manager (WordPress)
Information Technology
- 6 - 10 Yrs
- Ahmedabad
Location: Ahmedabad / Remote
Experience: 6+ Years
About Us:
E2M Solutions is home to a growing remote team of WordPress experts building innovative digital solutions. We pride ourselves on delivering consistent value and excellence to our client...
1 Day ago
Senior WordPress Frontend Developer
Information Technology
- 5 - 10 Yrs
- Ahmedabad
At E2M Solutions, we're building a powerhouse remote team to deliver high-performing, user-centric WordPress solutions. If you live and breathe frontend development and are looking to work on cutting-edge WordPress projects with a passionate global t...
1 Day ago
Python Developer - C++/EDA
Information Technology
- Chennai, Tamil Nadu, India
Job DescriptionWe are seeking a highly skilled C++ Python Developer with a strong background in software development, scripting, and EDA tool integration. This role focuses on creating, enhancing, and maintaining tools used in silicon design and ver...
1 Day ago
Business Analyst
Information Technology
- Chennai, Tamil Nadu, India
Project Role : Business AnalystProject Role Description : Analyze an organization and design its processes and systems, assessing the business model and its integration with technology. Assess current state, identify customer requirements, and defin...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in