The recall report
What the retrieval owner gets after every eval run
Not a recall number against the pool. The size of the pool, the size of the population it was drawn from, and recall against the second one.
Retrieval recall, corrected
The pool holds 985 relevant documents, every one of them surfaced by at least one retriever, so recall against it is 100% and cannot fall. BM25 found 737 and dense found 630; 382 were found by both. That overlap puts the relevant population at 1,215, with a 95% interval of 1,162 to 1,268. About 230 relevant documents are in the corpus and in no retriever’s output.
Against that population, BM25 recall is 60.7% rather than the 74.8% the pool reports, dense recall is 51.9% rather than 64.0%, and the union reaches 81.1%.
Caveat. The estimate assumes the two retrievers find documents independently. They do not: documents that match on words also tend to match on meaning, and that shared ease inflates the overlap. An inflated overlap makes the population look smaller than it is. Read 230 as the least that is missing, not the most.
What to do. Keep both retrievers: BM25 surfaces 355 relevant documents dense does not, and dense surfaces 248 that BM25 does not. To check the floor, label 48 queries in full, every candidate chunk rather than only the retrieved ones; that pins the missing share to within five points. To measure the dependence this report can only warn about, add a third retriever that works differently from both, such as a metadata or structured index, and compare the three pairwise estimates. If they agree, the floor is close to the truth. If they spread, the spread is the dependence.