Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documents

Document detail

ID

oai:arXiv.org:2405.02732

Topic

Computer Science - Computation and... Computer Science - Information Ret...

Author

Singhania, Sneha Razniewski, Simon Weikum, Gerhard

Year

2024

listing date

5/8/2024

Keywords

language recall

Metrics

Abstract

Methods for relation extraction from text mostly focus on high precision, at the cost of limited recall.

High recall is crucial, though, to populate long lists of object entities that stand in a specific relation with a given subject.

Cues for relevant objects can be spread across many passages in long texts.

This poses the challenge of extracting long lists from long texts.

We present the L3X method which tackles the problem in two stages: (1) recall-oriented generation using a large language model (LLM) with judicious techniques for retrieval augmentation, and (2) precision-oriented scrutinization to validate or prune candidates.

Our L3X method outperforms LLM-only generations by a substantial margin.

Singhania, Sneha,Razniewski, Simon,Weikum, Gerhard, 2024, Recall Them All: Retrieval-Augmented Language Models for Long Object List Extraction from Long Documents