@inproceedings{fazzioni-etal-2026-biatron,
    title = "Biatron: A Parameter-Efficient Small Language Model for {B}razilian {P}ortuguese with Integrated Mathematical Reasoning",
    author = "Fazzioni, Daniel  and
      Almeida, Maria C. X. de  and
      Moreira, Anna P. V. L. B.  and
      Soares, Anderson S.  and
      Oliveira, S{\'a}vio S. T. de  and
      Federson, Fernando M.",
    editor = "Souza, Marlo  and
      de-Dios-Flores, Iria  and
      Santos, Diana  and
      Freitas, Larissa  and
      Souza, Jackson Wilke da Cruz  and
      Ribeiro, Eug{\'e}nio",
    booktitle = "Proceedings of the 17th International Conference on Computational Processing of {P}ortuguese ({PROPOR} 2026) - Vol. 1",
    month = apr,
    year = "2026",
    address = "Salvador, Brazil",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-dnd/2026.propor-1.86/",
    pages = "868--877",
    ISBN = "979-8-89176-387-6",
    abstract = "The development of Small Language Models (SLMs) for Portuguese faces significant challenges in balancing parameter efficiency with specialized capabilities, particularly in mathematical reasoning domains where existing models demonstrate limited native competence. This work introduces the first model in the Biatron series, a 345-million-parameter language model specifically optimized for Brazilian Portuguese through strategic data curation rather than brute-force parameter scaling. Using a carefully designed 60-30-10 data mixture combining high-quality Portuguese text from GigaVerbo, chain-of-thought reasoning examples, and mathematical datasets, Biatron was trained on 300 billion tokens using the Megatron-LM framework, achieving 32{\%} Model FLOP Utilization.The model attains an overall score of 0.245 (aggregate performance) on Portuguese-specific benchmarks, approaching within 1.6{\%} of Tucano-630M{'}s performance while utilizing 45{\%} fewer parameters. Most significantly, Biatron achieves 7.5{\%} Pass@1 accuracy on mathematical reasoning tasks{---}more than doubling the performance of Tucano-2.4B (3.5{\%}) despite being nearly seven times smaller. These results validate that strategic data mixing can rival parameter scaling for language model development, establishing a reproducible methodology for efficient AI development in resource constrained language contexts. To support reproducibility and further research, the final model weights, training logs, and intermediate checkpoints are publicly available."
}