@inproceedings{olson-etal-2025-probing,
    title = "Probing Semantic Routing in Large Mixture-of-Expert Models",
    author = "Olson, Matthew Lyle  and
      Ratzlaff, Neale  and
      Hinck, Musashi  and
      Luo, Man  and
      Yu, Sungduk  and
      Xue, Chendi  and
      Lal, Vasudev",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2025",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-luhme/2025.findings-emnlp.991/",
    doi = "10.18653/v1/2025.findings-emnlp.991",
    pages = "18263--18278",
    ISBN = "979-8-89176-335-7",
    abstract = "In the past year, large ($>100$B parameter) mixture-of-expert (MoE) models have become increasingly common in the open domain. While their advantages are often framed in terms of efficiency, prior work has also explored functional differentiation through routing behavior. We investigate whether expert routing in large MoE models is influenced by the \textit{semantics} of the inputs. To test this, we design two controlled experiments. First, we compare activations on sentence pairs with a shared target word used in the same or different senses. Second, we fix context and substitute the target word with semantically similar or dissimilar alternatives. Comparing expert overlap across these conditions reveals clear, statistically significant evidence of \textit{semantic routing} in large MoE models."
}