Senior Data Engineer

Overview

We’re a seed funded stealth startup building in the intersection of AI and Securities market. We are based in Koramangala, Bangalore. Our founders have 50+ years of experience in Software and Financial Services and have built and sold software product companies before (repeat entrepreneurs).

As our Senior data engineer you’ll design the data pipelines, storage layers, and tooling that make all of that information trustworthy, query-able and lightning fast for our proprietary Retrieval Engine.

You might be a fit if you are:

• 4-6 years experience as a data engineer with exposure to large data pipelines
• Size of data handled – 50K-100K documents, with 1000’s of daily live updates
• Build scalable and reliable pipelines using leading commercial and open source tools like Databricks or Dragster / Temporal / Ray or similar
• Experience with building data pipeline for GenAI applications is major plus
• Graduate from B.tech. or similar programs.
• Strong desire to work in a startup and build a career as a software developer.
• Experience with Cloud technologies (Azure preferred)

Nice-to-have extras

• Past experience with handling Large scale financial data.
• Past experience in the search science (LLM based retrieval engine).

What you will be working on

• Building robust and performant backend services to power the next gen AI agents in the securities market.
• Building data pipelines to ingest, process and store high quality domain specific data.
• Building training pipeline for our proprietary LLM and ML models.

An overview of tech stack:

• webserver: FastAPI or Node.js
• Streaming systems: Kafka, Apache Flink
• Batch processing: Apache PySpark, Airflow
• Database: MongoDB, ElasticSearch, PostGreSQL, PGVectors, Qdrant Vector DB, graph DB etc.