Data Pipeline Engineer

Data Pipeline Engineer

Location: Onsite - Origin Medical Research Lab, 644, 12th Cross Road, HSR Layout, Bengaluru, Karnataka 560102

Team: Clinical Data Engineering

Job Type: Full-Time

Skills: Python & Data Libraries, ETL Development, Data Pipeline Architecture, Deployment & Monitoring, ELK Stack and Postgres database

Origin Medical Research Lab seeks highly talented and self-motivated individuals ready to build impactful products to democratize quality prenatal care globally for every expecting mother. At Origin Medical Research Lab, we offer an empowering work environment that allows you to take full ownership of your work and fosters collaboration and personal growth. We look for exceptional individuals and outliers who want to join us on this mission. Embark on a purpose-driven career journey by joining our team.

About Origin Medical Research Lab

Origin Medical Research Lab is the research arm of Origin Medical. Here, we strive to bring together the best and brightest minds at the intersection of AI and healthcare to fulfill Origin Medical’s mission.

By combining the knowledge of healthcare and AI, it is on a journey to build state-of-the-art solutions aimed at supporting a broad spectrum of healthcare providers in rural and urban communities, allowing them to practice at the top of their licenses. With AI in the imaging workflow, clinicians can more confidently deliver timely interventions, enhance pregnancy outcomes, identify high-risk pregnancies to reduce maternal mortality, and significantly lower infant mortality rates.

Origin Medical, headquartered in Cambridge, Massachusetts, USA, is driven by a mission to advance maternal health equity by improving access to quality prenatal care with artificial intelligence.

About the Role

As a Data Pipeline Engineer, you will be responsible for designing, building, and maintaining robust data pipelines that enable real-time AI analysis of fetal ultrasound images. This includes developing and optimizing data ingestion pipelines, ensuring minimal latency, and maintaining pipeline stability and reliability.

Design, Build, and Maintain Data Pipelines

• Architect robust pipelines for both real-time and batch data processing.
• Ensure scalability, reliability, and performance across data workflows.

Develop and Optimize Data Ingestion Pipelines

• Create high-throughput, low-latency pipelines tailored for AI and ML workloads.
• Continuously refine data flows to improve speed and efficiency.

Implement and Manage ETL Processes

• Design ETL workflows for seamless extraction, transformation, and loading of data.
• Integrate data from multiple sources to ensure consistency and accessibility.

Deploy, Monitor, and Troubleshoot Pipelines

• Set up monitoring and alerting systems to proactively detect issues.
• Perform root cause analysis and implement fixes for pipeline failures.

Collaborate with AI and Engineering Teams

• Work cross-functionally to understand data needs for model development and product features.
• Support experimentation and deployment of data-driven solutions.

Who are we looking for?

• Preferred Master’s or Bachelor’s degree in Computer Science, Software Engineering, or Information Technology.
• Strong proficiency in Python, including data structures, queues, threading, and multiprocessing (managed dictionaries, worker processes).
• Experience with essential data libraries such as NumPy and Pandas.
• Hands-on experience in building and maintaining ETL pipelines for data integration.
• Hands-on experience in architecting data pipelines is a plus.
• Proven experience in developing data ingestion pipelines for real-time and batch data processing.
• Experience with pipeline deployments, including automation and monitoring.
• Experience with the ELK Stack (Elasticsearch, Logstash, Kibana) for log management and analytics.
• Experience working with Postgres databases for data storage and retrieval.
• Knowledge of data engineering best practices for internal tools and scalable solutions.

Working at Origin Medical Research Lab

• You will receive competitive monthly compensation aligned with industry standards. Additionally, we provide a comprehensive benefits package, including: Provident fund, Paid annual leaves, Sick leaves, Wellness allowance, Insurance allowance.
• You will work with an exceptional team of highly qualified individuals who strive towards a common goal of delivering a product that improves the standard of care for expecting mothers everywhere.
• You will also collaborate with renowned clinicians, AI scientists, and business leaders from around the world.
• At Origin Medical Research Lab, we take pride in fostering an inclusive and optimistic company culture that places great value on collaboration, teamwork, and work-life balance. As a valued member of our team, you will have the opportunity to join a supportive environment where individuals genuinely care about each other's success and well-being. Our dedicated colleagues are always ready to lend a helping hand and wish you nothing but the best in your professional journey.