Chennai, Tamil Nadu, India
Information Technology
Full-Time
Positive Synergy
Overview
Job Description
Seeking a Lead Data Scientist (Generative AI) to spearhead the development of advanced AI-powered classification and matching systems on Databricks. You will contribute to flagship programs like the Diageo AI POC by building RAG pipelines, deploying agentic AI workflows, and scaling LLM-based solutions for high-precision entity matching and MDM modernization.
Key Responsibilities
Seeking a Lead Data Scientist (Generative AI) to spearhead the development of advanced AI-powered classification and matching systems on Databricks. You will contribute to flagship programs like the Diageo AI POC by building RAG pipelines, deploying agentic AI workflows, and scaling LLM-based solutions for high-precision entity matching and MDM modernization.
Key Responsibilities
- Design and implement end-to-end AI pipelines for product classification, fuzzy matching, and deduplication using LLMs, RAG, and Databricks-native workflows.
- Develop scalable, reproducible AI solutions within Databricks notebooks and job clusters, leveraging Delta Lake, MLflow, and Unity Catalog.
- Engineer Retrieval-Augmented Generation (RAG) workflows using vector search and integrate with Python-based matching logic.
- Build agent-based automation pipelines (rule-driven + GenAI agents) for anomaly detection, compliance validation, and harmonization logic.
- Implement explainability, audit trails, and governance-first AI workflows aligned with enterprise-grade MDM needs.
- Collaborate with data engineers, BI teams, and product owners to integrate GenAI outputs into downstream systems.
- Contribute to modular system design and documentation for long-term scalability and :
- Bachelors/Masters in Computer Science, Artificial Intelligence, or related field.
- 5-7 years of overall Data Science experience with 2+ years in Generative AI / LLM-based applications.
- Deep experience with Databricks ecosystem: Delta Lake, MLflow, DBFS, Databricks Jobs & Workflows.
- Strong Python and PySpark skills with ability to build scalable data pipelines and AI workflows in Databricks.
- Experience with LLMs (e.g., OpenAI, LLaMA, Mistral) and frameworks like LangChain or LlamaIndex.
- Working knowledge of vector databases (e.g., FAISS, Chroma) and prompt engineering for Exposure to MDM platforms (e.g., Stibo STEP) and familiarity with data harmonization challenges.
- Experience with explainability frameworks (e.g., SHAP, LIME) and AI audit tooling.
- Knowledge of agentic AI architectures and multi-agent orchestration. - Familiarity with Azure Data Hub and enterprise data ingestion frameworks. - Understanding of data governance, lineage, and regulatory compliance in AI systems.
Similar Jobs
View All
1 Day ago
Lead Python Developer
Information Technology
- 8 - 12 Yrs
- Anywhere in India/Multiple Locations
My Client is revolutionizing the hospitality industry by transforming how businesses manage their finances through cutting-edge Accounts Payable automation. Our mission? To eliminate manual work from business expenses and empower operators with real-...
1 Day ago
Business Analyst I, DeviceOS
Information Technology
- Chennai, Tamil Nadu, India
This job is with Amazon, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. DescriptionAmazon is looking for a data-savvy professional to crea...
1 Day ago
IT Support Mgr II
Information Technology
- Chennai, Tamil Nadu, India
This job is with Amazon, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. DescriptionAre you passionate about helping people solve IT proble...
1 Day ago
EverestDX - Senior Python Developer
Information Technology
- Chennai, Tamil Nadu, India
Job Title : Sr. Python DeveloperJob Location : Hyderabad Type : Full : 8+ Period : Immediate to 15 days joiners are highly : Design, develop, and maintain scalable Python applications and AI-powered solutions. Collaborate with cross-functional tea...
1 Day ago
.NET Application Developer
Information Technology
- Chennai, Tamil Nadu, India
🌟 We're Hiring: .NET Application Developer! 🌟We are looking for a skilled .NET Application Developer with at least 5 years of experience to join our dynamic team. The ideal candidate will have a strong background in developing robust applications us...
1 Day ago
Full Stack Developer - Node.js/React.js
Information Technology
- Chennai, Tamil Nadu, India
Designation : Full Stack Developer( React.js+ Node.js)Job Location : PrabhadeviJob Day & Timing : Mon - Fri , 9 AM -6 PMJob Description Perform hands-on computer program improvement, counting composing code and unit tests, doing verification of co...
1 Day ago
Sr. Test Engineer - QA Job
Information Technology
- Chennai, Tamil Nadu, India
We use cookies to offer you the best possible website experience. Your cookie preferences will be stored in your browser’s local storage. This includes cookies necessary for the website's operation. Additionally, you can freely decide and change any...
1 Day ago
QA Engineer
Information Technology
- Chennai, Tamil Nadu, India
Flowserve is a world-leading manufacturer and aftermarket service provider of comprehensive flow control systems. Join a company whose people are committed to building a more sustainable future to make the world better for everyone. With 16,000+ emp...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in