Fine-Tuning SmolLM2 for Unit Conversion Tasks

2025-02-152025-02-15 • 1 min read

Fine-Tuning SmolLM2 for Unit Conversion Tasks

For one of my grad school projects, I fine-tuned SmolLM2 to handle reasoning-based unit conversions.

Highlights:

• Used LoRA fine-tuning to reduce training overhead
• Evaluated model performance on conversion prompts
• Achieved ~50% accuracy and 85% answer rate with rejection sampling

This experiment gave me deeper insights into LLM reasoning limitations and ways to push small models further.