On the Theoretical Advantages of Bilinear Similarities in Dense Retrieval

Author

Shubham Chatterjee, Missouri University of Science and TechnologyFollow

Abstract

We present a theoretical and empirical study of bilinear similarity functions in neural IR, showing that they are strictly more expressive than dot-product and weighted dot-product (WDP) models under fixed embeddings. We prove this separation formally and illustrate it with the Structured Agreement Ranking Task, where a simple rank-2 bilinear model achieves 100% accuracy while all WDP models fail. This highlights the importance of modeling feature interactions for conditional relevance. On MS MARCO, low-rank bilinear models significantly outperform dot-product baselines: a rank-32 model triples performance (MRR@10: 0.090 vs. 0.031), and rank-128 approaches a 4x gain. These results offer a principled and practical case for using low-rank bilinear models in dense retrieval. Code:https://github.com/shubham526/bilinear-projection-theory.

Recommended Citation

S. Chatterjee, "On the Theoretical Advantages of Bilinear Similarities in Dense Retrieval," Lecture Notes in Computer Science, vol. 16134 LNCS, pp. 132 - 139, Springer, Jan 2026.

The definitive version is available at https://doi.org/10.1007/978-3-032-06069-3_11

Department(s)

Computer Science

International Standard Book Number (ISBN)

978-303206068-6

International Standard Serial Number (ISSN)

1611-3349; 0302-9743

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2026 Springer, All rights reserved.

Publication Date

01 Jan 2026