Jiawen Lyn
2025
TransLaTeX: Exposing the Last-Mile Execution Gap in LLM-Agent for Scientific Formatting
Jiawen Lyn
|
Yvette Graham
Proceedings of The First Workshop on Human–LLM Collaboration for Ethical and Responsible Science Production (SciProdLLM)
Large Language Models (LLMs) have achieved remarkable progress in tasks such as survey writing and language polishing, yet the final stage of LaTeX formatting and template adaptation remains a neglected and error-prone bottleneck.We identify an execution illusion, where LLMs produce linguistically fluent but unexecutable LaTeX code.To address this, we introduce TransLaTeX—the first reasoning-and-control framework that converts documents between scholarly templates with compiler-level verifiability.TransLaTeX achieves three key innovations:(1) Structure–content separation via placeholder masking, ensuring privacy and less token consumption;(2) SafeFormatBench, the first benchmark dedicated to executable LaTeX generation and template conversion; and(3) Execution-grounded verification across compilation, policy compliance, and visual consistency.TransLaTeX outperforms Pandoc and full-text LLM baselines on SafeFormatBench in compilation rate, ACL policy compliance, and layout fidelity, effectively mitigating the execution illusion.
2024
ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models
Zequan Liu
|
Jiawen Lyn
|
Wei Zhu
|
Xing Tian
|
Yvette Graham
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Parameter-efficient fine-tuning (PEFT) is widely studied for its effectiveness and efficiency in the era of large language models. Low-rank adaptation (LoRA) has demonstrated commendable performance as a popular and representative method. However, it is implemented with a fixed intrinsic rank that might not be the ideal setting for the downstream tasks. Recognizing the need for more flexible downstream task adaptation, we extend the methodology of LoRA to an innovative approach we call allocating low-rank adaptation (ALoRA) that enables dynamic adjustments to the intrinsic rank during the adaptation process. First, we propose a novel method, AB-LoRA, that can effectively estimate the importance score of each LoRA rank. Second, guided by AB-LoRA, we gradually prune abundant and negatively impacting LoRA ranks and allocate the pruned LoRA budgets to important Transformer modules needing higher ranks. We have conducted experiments on various tasks, and the experimental results demonstrate that our ALoRA method can outperform the recent baselines with comparable tunable parameters.