Abstract
As language technologies gain prominence in real-world settings, it is important to understand *how* changes to language affect reader perceptions. This can be formalized as the *causal effect* of varying a linguistic attribute (e.g., sentiment) on a reader’s response to the text. In this paper, we introduce Text-Transport, a method for estimation of causal effects from natural language under any text distribution. Current approaches for valid causal effect estimation require strong assumptions about the data, meaning the data from which one *can* estimate valid causal effects often is not representative of the actual target domain of interest. To address this issue, we leverage the notion of distribution shift to describe an estimator that *transports* causal effects between domains, bypassing the need for strong assumptions in the target domain. We derive statistical guarantees on the uncertainty of this estimator, and we report empirical results and analyses that support the validity of Text-Transport across data settings. Finally, we use Text-Transport to study a realistic setting—hate speech on social media—in which causal effects do shift significantly between text domains, demonstrating the necessity of transport when conducting causal inference on natural language.- Anthology ID:
- 2023.emnlp-main.82
- Volume:
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Houda Bouamor, Juan Pino, Kalika Bali
- Venue:
- EMNLP
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 1288–1304
- Language:
- URL:
- https://aclanthology.org/2023.emnlp-main.82
- DOI:
- 10.18653/v1/2023.emnlp-main.82
- Cite (ACL):
- Victoria Lin, Louis-Philippe Morency, and Eli Ben-Michael. 2023. Text-Transport: Toward Learning Causal Effects of Natural Language. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1288–1304, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Text-Transport: Toward Learning Causal Effects of Natural Language (Lin et al., EMNLP 2023)
- PDF:
- https://preview.aclanthology.org/emnlp-22-attachments/2023.emnlp-main.82.pdf