Bangalore, Karnataka, India
Information Technology
Full-Time
FiftyFive Technologies
Overview
Job Title: Mid Senior Data Engineer (LLM & AI Infrastructure)
About the Role
We are looking for a Data Engineer with a minimum of 1 year of experience working with LLMs and a strong background in data pipelines, storage optimization, and AI-driven data processing. The ideal candidate will play a key role in managing, optimizing, and scaling data architectures to support LLM applications and AI workflows.
Key Responsibilities
Design and develop scalable, high-performance data pipelines for AI and LLM-powered applications.- Work with structured and unstructured data, ensuring efficient preprocessing, transformation, and storage.
- Implement and optimize data retrieval and indexing strategies for LLM fine-tuning and inference.
- Manage vector databases (FAISS, Chroma, Pinecone, Weaviate, Astra DB.) for retrieval-augmented generation (RAG) workflows.
- Build and maintain ETL/ELT workflows using tools like Airflow, Prefect, or Dagster.
- Ensure data quality, governance, and lineage to support AI-driven insights.
- Collaborate with ML engineers and researchers to improve LLM data pipelines and infrastructure.
- Work with cloud-based data storage solutions (AWS S3, GCP BigQuery, Azure Data Lake, etc.).
- Automate data monitoring, validation, and debugging to ensure seamless pipeline execution.
Required Skills & Experience
4+ years of overall experience in data engineering, AI infrastructure, or related fields.- At least 1 year of hands-on experience working with LLMs and AI data workflows.
- Strong expertise in Python, SQL, and distributed data processing frameworks (Spark, Dask, Ray, or similar).
- Experience with vector databases and retrieval systems for AI-driven applications.
- Knowledge of data modeling, indexing, and storage optimization.
- Familiarity with ETL/ELT frameworks like Apache Airflow, Prefect, or Dagster.
- Experience handling large-scale datasets and optimizing data ingestion pipelines.
- Understanding of cloud-based data architectures (AWS, GCP, Azure).
- Basic knowledge of MLOps principles and integrating data workflows with AI models.
Nice to Have
Exposure to LLM fine-tuning, embeddings, and retrieval-augmented generation (RAG).- Experience with LangChain, Hugging Face Transformers, or OpenAI APIs.
- Familiarity with feature stores (Feast, Tecton) and streaming platforms (Kafka)
Similar Jobs
View All
2 Days ago
TrueFan - Senior Machine Learning Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
About UsTrueFan is at the forefront of AI-driven content generation, leveraging cutting-edge generative models to build next-generation products. Our mission is to redefine content generation space through advanced AI technologies, including deep ge...
2 Days ago
Salesforce commerce cloud consultant
Information Technology
- Thiruvananthapuram, Kerala, India
Salesforce Commerce Cloud consultant 5+ Years of Experience 6 to 12 months Mode - Remote 1.1LPM - 1.2LPM Max Key Responsibilities Translate business requirements into scalable Salesforce Service Cloud solutions, in collaboration with CAE's technic...
2 Days ago
Cloud Infrastructure Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
DescriptionInvent the future with us. Recognized by Fast Company’s 2023 100 Best Workplaces for Innovators List, Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focuse...
2 Days ago
Devops Engineer- Intermetiate
Information Technology
- Thiruvananthapuram, Kerala, India
BackJD: Dev ops Engineer:As a DevOps Specialist- should be able to take ownership of the entire DevOps process, including Automated CI/CD pipelines and deployment to production.They should also be comfortable with risk analysis and prioritization.Le...
2 Days ago
Sr Data Scientist (London)
Information Technology
- Thiruvananthapuram, Kerala, India
AryaXAI stands at the forefront of AI innovation, revolutionizing AI for mission-critical, highly regulated industries by building explainable, safe, and aligned systems that scale responsibly. Our mission is to create AI tools that empower research...
2 Days ago
Software Test Engineer
Information Technology
- Thiruvananthapuram, Kerala, India
By clicking the “Apply” button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda’s Privacy Notice and Terms of Use. I further att...
2 Days ago
Software Developer 5 (Java Fullstack)
Information Technology
- Thiruvananthapuram, Kerala, India
Job DescriptionBuilding off our Cloud momentum, Oracle has formed a new organization - Oracle Health Applications & Infrastructure. This team focuses on product development and product strategy for Oracle Health, while building out a complete platfo...
2 Days ago
Java Developer - Spring Frameworks
Information Technology
- Thiruvananthapuram, Kerala, India
Java DescriptionWe are looking for a passionate and talented Java Developer with 2-3 years of hands-on experience to join our growing development team.The ideal candidate should have a strong foundation in Java technologies and the ability to develo...
Talk to us
Feel free to call, email, or hit us up on our social media accounts.
Email
info@antaltechjobs.in