Research

How Small Can It Go? Parameter–Data Scaling of Romanized Nepali Small Language Models on a Consumer GPU

Abstract

Romanized Nepali is common in everyday digital writing, yet small open language models often struggle to generate it. This study asks how model size and fine-tuning data jointly affect that ability. We evaluated Qwen3 models with 0.6B, 4B, and 8B parameters across zero-shot, roughly 2,000-example, and roughly 13,000-example settings, using 4-bit QLoRA on a single RTX 4070 GPU. A 4B model trained on about 2,000 examples achieved 33.28 ChrF++, above a 30.7 acceptability threshold calibrated from human judgments, while using half the parameters and about one-seventh the data of the 8B full-data reference. Changes to the fine-tuning recipe had smaller effects than model and data scale; three-shot prompting without fine-tuning also approached the threshold. Because Romanized Nepali spelling is not standardized, further human validation of automatic metrics remains important.