Abstract
Current query expansion models use pseudo-relevance feedback to improve first-pass retrieval effectiveness; however, this fails when the initial results are not relevant. Instead of building a language model from retrieved results, we propose Generative Relevance Feedback (GRF) that builds probabilistic feedback models from long-form text generated from Large Language Models. We study the effective methods for generating text by varying the zero-shot generation subtasks: queries, entities, facts, news articles, documents, and essays. We evaluate GRF on document retrieval benchmarks covering a diverse set of queries and document collections, and the results show that GRF methods significantly outperform previous PRF methods. Specifically, we improve MAP between 5-19% and nDCG@10 17-24% compared to RM3 expansion and achieve state-of-the-art recall across all datasets.
Recommended Citation
I. Mackie et al., "Generative Relevance Feedback with Large Language Models," SIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2026 - 2031, Association for Computing Machinery, Jul 2023.
The definitive version is available at https://doi.org/10.1145/3539618.3591992
Department(s)
Computer Science
Keywords and Phrases
Document Retrieval; Pseudo-Relevance Feedback; Text Generation
International Standard Book Number (ISBN)
978-145039408-6
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2024 Association for Computing Machinery, All rights reserved.
Publication Date
19 Jul 2023
Comments
Engineering and Physical Sciences Research Council, Grant EP/V025708/1