@inproceedings{tamiosso-etal-2026-evaluating,
    title = "Evaluating Small Language Models for {E}nglish-to-{P}ortuguese Translation: Impact of Model Scale and Quantization",
    author = "Tamiosso, Gustavo Lopes  and
      Nunes, Rafael Oleques  and
      Balreira, Dennis Giovani",
    editor = "Souza, Marlo  and
      de-Dios-Flores, Iria  and
      Santos, Diana  and
      Freitas, Larissa  and
      Souza, Jackson Wilke da Cruz  and
      Ribeiro, Eug{\'e}nio",
    booktitle = "Proceedings of the 17th International Conference on Computational Processing of {P}ortuguese ({PROPOR} 2026) - Vol. 1",
    month = apr,
    year = "2026",
    address = "Salvador, Brazil",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-dnd/2026.propor-1.94/",
    pages = "943--952",
    ISBN = "979-8-89176-387-6",
    abstract = "Small language models (SLMs) are increasingly adopted for machine translation due to their lower computational and deployment costs, yet a focused and systematic evaluation for English-to-Portuguese remains limited. We benchmarked dozens of SLMs (135M{--}20B parameters) across multiple architectures and quantization schemes (FP16, Q8{\_}0, Q4{\_}K{\_}M) on two datasets: FLORES-101 (Portuguese subset, 1,012 sentences) and the multidomain OPUS-100 dataset ({\textasciitilde}10k sentences). We computed lexical and semantic metrics (BLEU, chrF, and BERTScore) and assessed statistical differences using non-parametric Friedman tests over paired sentence-level scores, followed by Wilcoxon signed-rank post-hoc comparisons with Holm correction. Normality assumptions are evaluated using the Shapiro{--}Wilk test. Our results strongly suggest that 8-bit quantization (Q8{\_}0) preserves semantic quality with negligible average loss, while 4-bit quantization (Q4{\_}K{\_}M) reaches statistical significance in roughly half of model configurations, paired effect sizes (Cliff{'}s {\ensuremath{\delta}}) remain negligible to small in magnitude, with measurable degradation concentrated in lower-capacity models. Model scale exhibits only a weak correlation with translation quality: medium-sized models can match or outperform larger ones depending on model family and pretraining. These findings highlight trade-offs between efficiency and quality and inform the design of practical English{--}to-Portuguese translation pipelines based on SLMs."
}