Da Yan


2023

pdf
Discovering Language Model Behaviors with Model-Written Evaluations
Ethan Perez | Sam Ringer | Kamile Lukosiute | Karina Nguyen | Edwin Chen | Scott Heiner | Craig Pettit | Catherine Olsson | Sandipan Kundu | Saurav Kadavath | Andy Jones | Anna Chen | Benjamin Mann | Brian Israel | Bryan Seethor | Cameron McKinnon | Christopher Olah | Da Yan | Daniela Amodei | Dario Amodei | Dawn Drain | Dustin Li | Eli Tran-Johnson | Guro Khundadze | Jackson Kernion | James Landis | Jamie Kerr | Jared Mueller | Jeeyoon Hyun | Joshua Landau | Kamal Ndousse | Landon Goldberg | Liane Lovitt | Martin Lucas | Michael Sellitto | Miranda Zhang | Neerav Kingsland | Nelson Elhage | Nicholas Joseph | Noemi Mercado | Nova DasSarma | Oliver Rausch | Robin Larson | Sam McCandlish | Scott Johnston | Shauna Kravec | Sheer El Showk | Tamera Lanham | Timothy Telleen-Lawton | Tom Brown | Tom Henighan | Tristan Hume | Yuntao Bai | Zac Hatfield-Dodds | Jack Clark | Samuel R. Bowman | Amanda Askell | Roger Grosse | Danny Hernandez | Deep Ganguli | Evan Hubinger | Nicholas Schiefer | Jared Kaplan
Findings of the Association for Computational Linguistics: ACL 2023

As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data sources (which are not always available). Here, we automatically generate evaluations with LMs. We explore approaches with varying amounts of human effort, from instructing LMs to write yes/no questions to making complex Winogender schemas with multiple stages of LM-based generation and filtering. Crowdworkers rate the examples as highly relevant and agree with 90-100% of labels, sometimes more so than corresponding human-written datasets. We generate 154 datasets and discover new cases of inverse scaling where LMs get worse with size. Larger LMs repeat back a dialog user’s preferred answer (“sycophancy”) and express greater desire to pursue concerning goals like resource acquisition and goal preservation. We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse. For example, RLHF makes LMs express stronger political views (on gun rights and immigration) and a greater desire to avoid shut down. Overall, LM-written evaluations are high-quality and let us quickly discover many novel LM behaviors.

2021

pdf
Adversarial Attack against Cross-lingual Knowledge Graph Alignment
Zeru Zhang | Zijie Zhang | Yang Zhou | Lingfei Wu | Sixing Wu | Xiaoying Han | Dejing Dou | Tianshi Che | Da Yan
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

Recent literatures have shown that knowledge graph (KG) learning models are highly vulnerable to adversarial attacks. However, there is still a paucity of vulnerability analyses of cross-lingual entity alignment under adversarial attacks. This paper proposes an adversarial attack model with two novel attack techniques to perturb the KG structure and degrade the quality of deep cross-lingual entity alignment. First, an entity density maximization method is employed to hide the attacked entities in dense regions in two KGs, such that the derived perturbations are unnoticeable. Second, an attack signal amplification method is developed to reduce the gradient vanishing issues in the process of adversarial attacks for further improving the attack effectiveness.