Paul He
2025
CCG Revisited: A Multilingual Empirical Study of the Kuhlmann-Satta Algorithm
Paul He
|
Gerald Penn
Proceedings of the 18th International Conference on Parsing Technologies (IWPT, SyntaxFest 2025)
We revisit the polynomial-time CCG parsing algorithm introduced by Kuhlmann & Satta (2014), and provide a publicly available implementation of it. We evaluate its empirical performance against a naive CKY-style parser across the Parallel Meaning Bank (PMB) corpus. While the fast parser is slightly slower on average, relative to the size of the PMB, but the trend improves as a function of sentence length, and the PMB is large enough to witness an inversion. Our analysis quantifies this crossover and highlights the importance of derivational context decomposition in practical parsing scenarios.
Pointwise Mutual Information as a Performance Gauge for Retrieval-Augmented Generation
Tianyu Liu
|
Jirui Qi
|
Paul He
|
Arianna Bisazza
|
Mrinmaya Sachan
|
Ryan Cotterell
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Recent work suggests that large language models enhanced with retrieval-augmented generation are easily influenced by the order in which the retrieved documents are presented to the model when solving tasks such as question answering (QA).However, there is no method to date that exploits this phenomenon to improve generation.To fill this gap, in this study, we show that the pointwise mutual information between a context and a question is an effective gauge for language model performance.Importantly, this gauge does not depend on knowing the answer to the question a priori.Through experiments on two question-answering datasets using a variety of large language models, we find evidence for an empirical correlation between answer accuracy and pointwise mutual information.Additionally, we propose two methods that use the pointwise mutual information between a document and a question as a gauge for selecting and constructing prompts that lead to better performance, whose effectiveness we demonstrate through experimentation.
Search
Fix author
Co-authors
- Arianna Bisazza 1
- Ryan Cotterell 1
- Tianyu Liu 1
- Gerald Penn 1
- Jirui Qi 1
- show all...