An experiment fine-tuning a tiny 0.6B parameter LLM (Qwen 3:0.6B) for household question categorization showed a significant accuracy improvement over the baseline prompting-only approach. The project demonstrates that with frameworks like Unsloth and QLoRA, even very small models can be effectively adapted for specific classification tasks using a modest dataset.
An experiment was conducted to fine-tune a tiny language model (Qwen 3:0.6B) as a question classifier for a household chatbot project. The goal was to improve metadata-aware vector search by pre-categorizing questions.
To enhance RAG accuracy, a pre-processing step was introduced to categorize questions into known metadata groups (e.g., pool, car, HVAC). The baseline performance of the 0.6B model using only prompting was poor (~10% accuracy). The experiment aimed to significantly improve this using fine-tuning with Unsloth and QLoRA.
This project demonstrates that even models with very few parameters can be effectively specialized for specific domain classification tasks. It suggests a practical approach for building cost-effective and privacy-friendly AI agents on local hardware by fine-tuning tiny models for specific sub-tasks.