Free cookie consent management tool by TermsFeed Azure Data Engineer - ETL/Data Pipeline | Antal Tech Jobs
Back to Jobs
1 Day ago

Azure Data Engineer - ETL/Data Pipeline

decor
Bangalore, Karnataka, India
Information Technology
Full-Time
Deqode

Overview

Job Description

We are seeking a Senior Software Engineer (SSE) with strong expertise in Kafka, Python, and Azure Databricks to lead and contribute to our healthcare data engineering initiatives. This role is pivotal in building scalable, real-time data pipelines and processing large-scale healthcare datasets in a secure and compliant cloud environment.

The ideal candidate will have a solid background in real-time streaming, big data processing, and cloud platforms, along with strong leadership and stakeholder engagement capabilities.

Key Responsibilities

  • Design and develop scalable real-time data streaming solutions using Apache Kafka and Python.
  • Architect and implement ETL/ELT pipelines using Azure Databricks for both structured and unstructured healthcare data.
  • Optimize and maintain Kafka applications, Python scripts, and Databricks workflows to ensure performance and reliability.
  • Ensure data integrity, security, and compliance with healthcare standards such as HIPAA and HITRUST.
  • Collaborate with data scientists, analysts, and business stakeholders to gather requirements and translate them into robust data solutions.
  • Mentor junior engineers, perform code reviews, and promote engineering best practices.
  • Stay current with evolving technologies in cloud, big data, and healthcare data standards.
  • Contribute to the development of CI/CD pipelines and containerized environments (Docker, Kubernetes).

Required Skills & Qualifications

  • 4+ years of hands-on experience in data engineering roles.
  • Strong proficiency in Kafka (including Kafka Streams, Kafka Connect, Schema Registry).
  • Proficient in Python for data processing and automation.
  • Experience with Azure Databricks (or readiness to ramp up quickly).
  • Solid understanding of cloud platforms, with a preference for Azure (AWS/GCP is a plus).
  • Strong knowledge of SQL and NoSQL databases; data modeling for large-scale systems.
  • Familiarity with containerization tools like Docker and orchestration using Kubernetes.
  • Exposure to CI/CD pipelines for data applications.
  • Prior experience with healthcare datasets (EHR, HL7, FHIR, claims data) is highly desirable.
  • Excellent problem-solving abilities and a proactive mindset.
  • Strong communication and interpersonal skills to work in cross-functional teams.

(ref:hirist.tech)
Share job
Similar Jobs
View All
5 Hours ago
Head of AI Cybersecurity
Information Technology
  • 8 - 13 Yrs
  • Chennai, Mumbai (All Areas)
Location: Mumbai & Chennai- Hybrid (1 week WFO in a month) | Experience: 8+ years Type: Full-time Seeking a visionary and hands-on Head of AI Cybersecurity to lead the end-to-end security of AI/ML systems. This role will drive strategy, implement...
decor
1 Day ago
Full Stack Developer (React + Python + AWS)
Information Technology
  • 5 - 8 Yrs
  • Pune
About the Client: They are a Global leader in delivering cutting-edge inflight entertainment and connectivity (IFEC) solutions. About the Role: Roles and Responsibilites Lead Frontend Development (70%): Design, build, and optimize inte...
decor
1 Day ago
Senior Full Stack Software Engineer
Information Technology
  • 5 - 10 Yrs
  • Bangalore
Job Title: Senior Full Stack Software Engineer Location: Bengaluru Experience: 5+ Years Job Type: Full Time What our client offer Our client believes in rewarding our employees for their hard work. Our client offer competitive sala...
decor
1 Day ago
Technical Delivery Engineer
Information Technology
  • 10 - 15 Yrs
  • Bangalore, Gurgaon / Gurugram
Location: Bangalore / Gurugram Mode: Hybrid (2–3 days in office per week) Experience Required: 10+ years Employment Type: Permanent Role Overview As a Technical Delivery Engineer, you will be a key member of our infrastructure delivery t...
decor
1 Day ago
Lead Endur Specialist
Information Technology
  • 13 - 20 Yrs
  • Bangalore
What we offer: Our Client, believes in rewarding our employees for their hard work. We offer competitive salaries, company pensions and performance related benefits. Our people can also take advantage of our extensive flexible benefits package and...
decor
1 Day ago
Associate Data Scientist in Gurgaon, Jaipur
Information Technology
  • Bangalore, Karnataka, India
Key Responsibilities Utilize data analysis tools and software to extract meaningful insights from raw data. Collaborate with data engineers to design and implement databases, data collection systems, and analytical frameworks for optimized statist...
decor
1 Day ago
Eloelo - Android Developer - Mobile App Integration
Information Technology
  • Bangalore, Karnataka, India
Are you ready to be a part of the dynamic world of live streaming and social gaming? Look no further! Eloelo, an innovative Indian platform founded in February 2020 by ex-Flipkart veteran Saurabh Pandey, is on the lookout for passionate individuals ...
decor
1 Day ago
Senior Business Analyst
Information Technology
  • Bangalore, Karnataka, India
Backdrop AVIZVA is a Healthcare Technology Organization that harnesses technology to simplify, accelerate, & optimize the way healthcare enterprises deliver care. Established in 2011, we have served as strategic enablers for healthcare enterprises,...
decor

Talk to us

Feel free to call, email, or hit us up on our social media accounts.
Social media