Data Engineer - II

Overview

We are looking for a Data Engineer - a hands-on problem solver who loves experimenting with data and AI technologies, building quick prototypes, and testing cutting-edge ideas. The ideal candidate is someone with a portfolio of data engineering projects and a knack for rapid prototyping. The ideal candidate will possess strong analytical skills, expert-level proficiency in SQL and Python.

Responsibilities:

1. Quickly prototype AI-based solutions to test hypotheses and explore new ideas
2. Experiment with different AI models (LLMs, diffusion models, classic ML, etc.) to solve real-world problems
3. Fine-tune, adapt, and deploy open-source AI models for custom applications
4. Stay updated on the latest AI research and tools, and apply them in practical ways
5. Collaborate with application engineers, designer, and the product team to turn prototypes into scalable solutions

Skills (The ideal candidate will meet the following skills criteria):

1. 2+ years of experience but a strong portfolio of data engineering projects - GitHub, blog posts, demos, etc.
2. Expert knowledge of SQL and basic understanding of data pipeline and workflow orchestration tools such as DBT and Prefect
3. Hands-on experience with Python and AI frameworks - PyTorch, TensorFlow, Hugging Face, LangChain, etc.
4. Ability to quickly experiment with APIs (OpenAI, Anthropic, Mistral, etc.) and open-source models
5. Experience in rapid prototyping (Jupyter notebooks, Gradio, Streamlit, etc.).
6. Comfortable working with at least one of the cloud platforms (AWS, GCP, Azure)
7. A hacker mindset-willing to try, fail, iterate, and learn fast
8. Basic knowledge of autonomous agents, LLM tooling

Qualification & Experience:

1. Bachelor/Master/Engineering degree in IT/Computer Science/software engineering or relevant field
2. Experience with developing production-grade Python code
3. Experience with relational SQL and NoSQL databases