Ellipsis-dependent Reasoning: A New Challenge for Large Language Models

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningpeer review

95 Downloads (Pure)

Abstract

We propose a novel challenge for large language models: ellipsis-dependent reasoning. We define several structures of paired examples, where an ellipsis example is matched to its non-ellipsis counterpart, and a question is posed which requires resolution of the ellipsis. Test results show that the best models perform well on non-elliptical examples but struggle with all but the simplest ellipsis structures.
OriginalsprogEngelsk
TitelProceedings of the 61st Annual Meeting of the Association for Computational Linguistics
RedaktørerAnna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Vol/bind2
UdgivelsesstedStroudsburg
ForlagAssociation for Computational Linguistics
Publikationsdato2023
Sider39-47
ISBN (Trykt)9781959429715
StatusUdgivet - 2023
BegivenhedThe 61st Annual Meeting of the Association for Computational Linguistics - Westin Harbour Castle, Toronto, Canada
Varighed: 9 jul. 202314 jul. 2023
Konferencens nummer: 61
https://2023.aclweb.org/

Konference

KonferenceThe 61st Annual Meeting of the Association for Computational Linguistics
Nummer61
LokationWestin Harbour Castle
Land/OmrådeCanada
ByToronto
Periode09/07/202314/07/2023
Internetadresse

Citationsformater