Data Scientist - LLM - BluemachineAI

Overview

We are looking for a hands-on Senior Data Scientist / Research Scientist who can own end-to-end training and fine-tuning of open-source Large Language Models (LLMs)—from data curation and experimentation to evaluation and production deployment.

This is a builder role, not a script-runner role. You will work on Indian languages and code-switching (Hinglish, etc.), improve instruction following and tool/function calling reliability, and optimize models for low latency and high throughput in production.

Key Responsibilities

Model Training & Fine-Tuning

• Train and fine-tune open-source LLMs using:
• Continued pre-training, SFT, preference optimization (DPO / IPO / ORPO)
• Full fine-tuning, LoRA / QLoRA based approaches
• Improve model performance on:
• Indian languages and multilingual / code-mixed inputs
• Strong instruction following
• Reliable tool/function calling with structured JSON outputs
  

• Build and maintain high-quality training pipelines for:
• Instruction datasets, tool-call traces, multilingual corpora
• Synthetic data generation
• Implement:
• De-duplication, contamination checks
• Quality, safety, and PII filtering
  

• Design evaluation frameworks and dashboards:
• Offline and online evaluation, regression testing
• Tool-calling accuracy, schema validity, multilingual benchmarks
• Latency, throughput, and cost metrics
  

• Optimize models for production:
• Quantization (AWQ / GPTQ / bits-and-bytes)
• Distillation, speculative decoding, KV-cache optimization
• Deploy and serve models using:
• vLLM, TGI, TensorRT-LLM, ONNX (as applicable)
  

• Reduce hallucinations and improve refusal behavior
• Enforce deterministic and structured outputs
• Apply prompting + training strategies for robust compliance
  
  

• Work closely with engineering teams on:
• Model packaging, CI-based evaluation, A/B testing
• Monitoring quality drift in production
• Read research papers, propose experiments, and convert ideas into measurable improvements
  

• 4–6 years of experience in ML / Data Science with hands-on LLM training & fine-tuning
• Proven ability to drive end-to-end model improvement: Data → Training → Evaluation → Production constraints → Iteration
• Strong understanding of:
• Transformers, tokenization, multilingual modeling
• Fine-tuning methods: LoRA / QLoRA, full fine-tuning, continued pre-training
• Alignment techniques: SFT, DPO / IPO / ORPO
• Experience building or improving tool/function calling reliability
• Strong coding skills in Python, deep experience with PyTorch
• Experience with distributed training:
• DeepSpeed / FSDP / Accelerate
• Multi-GPU / multi-node setups
• Solid ML fundamentals: optimization, regularization, error analysis, scaling intuition
  

• Experience with Indian language NLP:
• Indic scripts, transliteration, normalization, code-mixing
• Experience with large-scale pre-training or continued pre-training
• Practical serving experience:
• vLLM, TGI, TensorRT-LLM
• Quantization calibration and performance profiling
• Exposure to data governance, privacy, and dataset documentation
  

• Modeling & Training: PyTorch, Hugging Face Transformers & Datasets, PEFT
• Distributed Training: DeepSpeed, FSDP, Accelerate
• Experiment Tracking: Weights & Biases, MLflow
• Serving: vLLM, TGI, TensorRT-LLM
• Infra: Docker, Kubernetes
• Optional: Ray, Airflow, Spark
• Bonus: Vector DB / RAG stack familiarity
  
  

• Ship a fine-tuned open-source LLM with measurable improvements in:
• Instruction following and tool-calling correctness
• Indian language and code-switching performance
• Lower latency and higher throughput at comparable quality
• Build a repeatable training + evaluation pipeline:
• Dataset versioning, training recipes, evaluation harness, regression gates
• Define a roadmap for future improvements:
• Distillation, preference tuning, multilingual expansion
  

['LLM Fine-Tuning', 'LLM training', 'NLP', 'Python']

Additional Information

Interview Process

• 30-min Intro & Role Fit
• Technical Deep Dive (LLM training, evals, production constraints)
• Take-Home / Live Exercise:
• Design an LLM fine-tuning + evaluation plan for tool calling & Indic languages
• Systems Round:
• Training vs serving trade-offs, cost/latency, failure modes
• Culture & Collaboration Round