@inproceedings{tekin-etal-2026-h3fusion,
    title = "{H}3{F}usion: Helpful, Harmless, Honest Fusion of Aligned {LLM}s",
    author = "Tekin, Selim Furkan  and
      Ilhan, Fatih  and
      Hu, Sihao  and
      Huang, Tiansheng  and
      Xu, Yichang  and
      Yahn, Zachary  and
      Liu, Ling",
    editor = "Demberg, Vera  and
      Inui, Kentaro  and
      Marquez, Llu{\'i}s",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://preview.aclanthology.org/ingest-eacl/2026.eacl-long.329/",
    pages = "6993--7013",
    ISBN = "979-8-89176-380-7",
    abstract = "The alignment of pre-trained LLMs continues to draw significant attention from both industry and academia, aiming to ensure responses that are helpful, harmless, and honest. However, identifying a point in the model{'}s representation subspace that simultaneously satisfies all these properties remains challenging. H3Fusion addresses this challenge by introducing a mixture-of-experts (MoE)-based fusion mechanism that models alignment as a controllable drift within the subspace, guided by a drift-regularization loss to balance competing alignment dimensions. Furthermore, we formulate the alignment by finding a dual objective of harnessing the distance of generated embeddings and alignment embeddings, and introduce gating loss by canalizing the activations on the contributing experts. Extensive evaluations of three benchmark datasets show that H3Fusion is more helpful, less harmful, and more honest in three aspects: it outperforms each individually aligned model by 11.37{\%}, and provides stronger robustness compared to the state-of-the-art LLM ensemble approaches by 13.77{\%} and model-merging approaches by 6.18 {\%}. Code is available at https://github.com/git-disl/h3fusion."
}