Bangalore, Karnataka, India
Information Technology
Full-Time
Oblige IT
Overview
Key Responsibilities
- Design, develop, and optimize large-scale data pipelines using PySpark and Apache Spark.
- Build scalable and robust ETL workflows leveraging AWS services such as EMR, S3, Lambda, and Glue.
- Collaborate with data scientists, analysts, and other engineers to gather requirements and deliver clean, well-structured data solutions.
- Integrate data from various sources, ensuring high data quality, consistency, and reliability.
- Manage and schedule workflows using Apache Airflow.
- Work on ML model deployment pipelines using tools like SageMaker and Anaconda.
- Write efficient and optimized SQL queries for data processing and validation.
- Develop and maintain technical documentation for data pipelines and architecture.
- Participate in Agile ceremonies, sprint planning, and code reviews.
- Troubleshoot and resolve issues in production environments with minimal supervision.
- Bachelor's or Masters degree in Computer Science, Engineering, or a related field.
- Python
- PySpark
- SQL
- AWS (EMR, EC2, S3, Lambda, Glue)
- Experience in developing and orchestrating pipelines using Apache Airflow.
- Familiarity with SageMaker for ML deployment and Anaconda for environment management.
- Proficiency in working with large datasets and optimizing Spark jobs.
- Experience in building data lakes and data warehouses on AWS.
- Strong understanding of data governance, data quality, and data lineage.
- Excellent documentation and communication skills.
- Comfortable working in a fast-paced Agile environment.
- Experience with Kafka or other real-time streaming platforms.
- Familiarity with DevOps practices and tools (e.g., Terraform, CloudFormation).
- Exposure to NoSQL databases such as DynamoDB or MongoDB.
- Knowledge of data security and compliance standards (GDPR, HIPAA).
- Work with cutting-edge technologies in a collaborative and innovative environment.
- Opportunity to influence large-scale data infrastructure.
- Competitive salary, benefits, and professional development support.
- Be part of a growing team solving real-world data challenges.
Similar Jobs
View All
1 Day ago
Program Manager
Information Technology
- 15 - 18 Yrs
- Gurgaon / Gurugram
We’re Hiring | Program Manager
Location: Gurugram, India
Domain: Program Management | Digital Systems | Java/.NET | Agile | Fintech/Supply Chain
Are you a strategic thinker with deep experience in program management of digital products? We’re lo...
1 Day ago
Technical Fullstack Architect - Node.js
Information Technology
- 50,00,000 - 60,00,000 INR - Annual
- 12 - 18 Yrs
- Hyderabad
About the Role:
We are seeking a Fullstack Technical Architect with deep expertise in backend development using Node.js and proficiency in frontend technologies like React or any modern JavaScript framework. You will play a key role in building an...
1 Day ago
Principal Engineer - Fullstack
Information Technology
- 30,00,000 - 40,00,000 INR - Annual
- 8 - 12 Yrs
- Mumbai
Looking for candidate who is enthusiastic to work in a Startup environment and build things from Scratch individually
Candidate has past experience in scalable consumer facing applications managing latency and traffic FullStack Individual Contribu...
1 Day ago
Asst. Manager / Dy. Manager – Talent Acquisition
Automotive
- 4,00,000 - 8,00,000 INR - Yearly
- 4 - 8 Yrs
- West Bengal
We are looking for dynamic and experienced professionals for the Talent Acquisition team at our Kharagpur Plant location. The incumbent will be responsible for managing the full-cycle recruitment process for both technical and functional roles across...
1 Day ago
Software Developer
Cybersecurity & Privacy
- 2 - 5 Yrs
- Gujarat, Maharashtra
Summary role description:
Hiring for a Solution Architect – Databricks for global technology consulting and system integration firm specializing in data engineering, AI and ML.
Company description:
Our client is a US-headquartere...
2 Days ago
Axis My India - Data Analyst - Python/Pandas
Pharmaceuticals
- Hyderabad, Telangana, India
About The CompanyAxis My India is Indias foremost Consumer Data Intelligence Company, which in partnership with Google is building a single-stop People Empowerment Platform, the a app, that aims to change peoples awareness, accessibility, and utiliz...
2 Days ago
Data Engineer
Pharmaceuticals
- Hyderabad, Telangana, India
Job Title - Data EngineerAbout TazapayTazapay is a cross border payment service provider. They offer local collections via local payment methods, virtual accounts and cards in over 70 markets. The merchant does not need to create local entities anyw...
2 Days ago
Master Data Analyst – Application Support
Pharmaceuticals
- Hyderabad, Telangana, India
About Inchcape Shipping ServicesAt Inchcape, our vision is to have a connected world, in which our customers trade successfully and make better decisions in every port, everywhere. We use technology and our global network to help our partners connec...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in