Cartio : Fine-tuning LLaMA 3 for an E-commerce Conversational Chatbot
Python
LLM
Fine-tune
Llama 3 - 8B
LoRA
Project Overview
- In an e-commerce setting, a chatbot should not just answer queries but also engage in interactive dialogue — asking clarifying questions, guiding the user through product searches, and handling follow-ups effectively.
- When we tested the standard LLaMA 3 model with prompt engineering, it often confused user and assistant roles, generated irrelevant responses, and skipped essential follow-up questions. For example, instead of asking "What product are you looking for?", it would directly generate an answer or hallucinate a response. This behavior made the chatbot unsuitable for a real-world e-commerce use case.
Approach to the Solution
- Requirement Analysis: Identified the gap between generic model outputs and the specific needs of e-commerce conversations (e.g., product recommendations, order tracking, returns, etc.).
- Dataset Preparation: Curated and structured a domain-specific customer support dataset containing real-world e-commerce queries and responses. This dataset emphasized follow-up questions, context retention, and appropriate responses.
-
Fine-tuning Strategy:
- Chose LoRA (Low-Rank Adaptation) for parameter-efficient fine-tuning.
-
Trained the model on NVIDIA RTX 4090 GPU, ensuring faster experimentation and
optimization.
Nvidia RTX 4090 GPU Details - Adjusted hyperparameters to balance response accuracy and conversational flow.
- Training: Fine-tuned LLaMA 3 - 8B on the prepared dataset until the model learned to correctly distinguish between user queries and assistant roles.
- Evaluation: Compared the before vs. after performance by testing with real e-commerce queries.
Results
- The fine-tuned chatbot demonstrated a significant improvement in conversation quality.
- Base model response issue, it just focuses on the generating next token without focusing on the user question and eventually ended up doing user and assistant conversation both from it side.
-
After fine-running with e-commerce based conversational dataset it was able to process and
understand correctly and even asking correctly follow up question.
Fine tuned model response for same Question : I want to cancel my order #12345. Can you help me? - It maintained context over multiple turns, improving the user experience.
- Responses became more accurate, natural, and aligned with e-commerce workflows like browsing, order status checks, and returns.
Base model Response for Question : I want to cancel my order #12345. Can you help me?
Impact
- Enabled a customer-friendly chatbot that can handle complex conversations without human intervention.
- Improved user satisfaction through interactive, context-aware dialogue.
- Reduced support workload by automating common queries like order status and returns.
- Showcased how domain-specific fine-tuning can transform a general LLM into a business-ready solution.