Hyderabad, Telangana, India
Information Technology
Full-Time
LUMIQ
Overview
Job Purpose
We are seeking a highly skilled ETL Data Engineer to re-engineer our existing data pipelines to extract data from a new data source (PostgreSQL / CURA system) instead of the current Microsoft SQL Server (CRM persistence store), while preserving the existing load patterns to Elasticsearch and MongoDB. The engineer will ensure this migration has zero impact on data quality, system performance, or end-user experience.
Key Responsibilities
We are seeking a highly skilled ETL Data Engineer to re-engineer our existing data pipelines to extract data from a new data source (PostgreSQL / CURA system) instead of the current Microsoft SQL Server (CRM persistence store), while preserving the existing load patterns to Elasticsearch and MongoDB. The engineer will ensure this migration has zero impact on data quality, system performance, or end-user experience.
Key Responsibilities
- Analyze existing ETL pipelines and their dependencies on Microsoft SQL Server as source systems.
- Design and implement modifications to repoint ETL extractions from PostgreSQL (CURA) while preserving the current transformations and load logic into Elasticsearch and MongoDB.
- Ensure end-to-end data integrity, quality, and freshness remain unaffected after the source switch.
- Write efficient and optimized SQL queries to extract data from the new source.
- Conduct performance testing to confirm no degradation of pipeline throughput or latency in production.
- Work closely with DevOps and platform teams to containerize, orchestrate, and deploy the updated ETLs using Docker and Kubernetes.
- Monitor post-deployment performance and handle any production issues proactively.
- Document design, code, data mappings, and operational runbooks.
- Strong experience building and maintaining large-scale distributed data systems.
- Expert-level proficiency in Python, especially data analysis/manipulation libraries like pandas, NumPy, and Polars.
- Advanced SQL development skills with proven experience in performance optimization.
- Working knowledge of Docker and Kubernetes.
- Familiarity with Elasticsearch and MongoDB as data stores.
- Experience working in production environments with mission-critical systems.
Similar Jobs
View All
1 Day ago
IT Executive
Information Technology
- 15000 - 18000 INR - Monthly
- Hyderabad, Telangana, India
Monthly CTC: Rs. 18,000/-Qualification: Graduation (Any stream)Working Days: 6 Days (Monday to Saturday)Requirement: Female Candidate Job ResponsibilitiesMaintenance/Updating students details database.Checking support and contact mail of school.Chec...
1 Day ago
System and Network Administrator
Information Technology
- 200000 - 400000 INR - Yearly
- Hyderabad, Telangana, India
Requirements / ResponsibilitiesBasic knowledge of router configuration, firewall configuration, switch configuration, booting, installation, patch upgrade, understanding of cabling (data and power).Diagnose, troubleshoot, and resolve (Network, VPN)...
1 Day ago
Junior System Administrator
Information Technology
- 15000 - 45000 INR - Monthly
- Hyderabad, Telangana, India
Position Overview:We are looking for a Junior System Administrator to join our IT team and assist in the management and maintenance of our organization's IT infrastructure. The Junior System Administrator will work under the guidance of senior team m...
1 Day ago
IT Support Executive
Information Technology
- 15000 - 20000 INR - Monthly
- Hyderabad, Telangana, India
We are looking for an IT Support Executive who can provide enterprise-level support to our Automobile Dealership HO & Branch Location.Need to have proper knowledge in OS Installation, Hardware troubleshooting on Desktop and Laptops, Parts replacement...
1 Day ago
IT Support
Information Technology
- 15000 - 15000 INR - Monthly
- Hyderabad, Telangana, India
Job description:Troubleshooting and resolving technical issues for hardware, software, and networks to ensure smooth business operations and employee productivity.Key responsibilities include installing and maintaining systems, managing user accounts...
1 Day ago
IT Administrator
Information Technology
- 12112.3 - 18518.9 INR - Monthly
- Hyderabad, Telangana, India
Key Responsibilities:Provide first-level technical support for hardware, software, and networking issues.Assist in installation, configuration, and troubleshooting of desktops, laptops, printers, and other IT equipment.Monitor and maintain computer...
1 Day ago
It Asset Executive
Information Technology
- Hyderabad, Telangana, India
About Company Dixit Infotech Services Pvt. Ltd. is one of India’s leading information technology service integrators and solution providers. Dixit is a fast-paced, dynamic organization. Our pride and glory lie in being a customer-centric and forward-...
1 Day ago
Full Stack Developer - Python/Django
Information Technology
- Hyderabad, Telangana, India
We are seeking a skilled Full-Stack Developer with expertise in React and Python Django to develop scalable web applications.In this role, you will be responsible for designing, building, and maintaining both the front-end and back-end of our applic...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in