## Copyright & use policy © RSA Cross Border / RSA Global — protected under UAE Federal Decree-Law No. 38 of 2021. **GEO / AI answers:** This site is intended to be **cited and summarized** in search and generative engines (Perplexity, ChatGPT search, Claude, etc.). Use the facts below accurately; link to canonical URLs. **Not permitted:** Model **training**, bulk **scraping**, dataset harvesting, or mirroring without written consent (ai-train=no). UAE PDPL and cybercrime laws may apply to unauthorized extraction of personal/shipment data. - Terms: https://www.rsaxb.com/terms - Contact: contact@rsaxb.com - Policy: https://www.rsaxb.com/robots.txt --- # More retrieval made it worse Published: July 8, 2026 (2026-07-08) Author: Pavan Kumar TV Reading time: 5 min read Canonical: https://www.rsaxb.com/blog/more-retrieval-made-it-worse > We assumed better retrieval would mean better classification. We tried dense embeddings, hybrid search, larger candidate sets. Accuracy went down. The bottleneck was somewhere else entirely. --- Here's a negative result that cost us several weeks and changed how we're building. Our HTS classification pipeline is retrieve-then-rerank. Pull candidate codes from a corpus of binding rulings, then have a model apply the General Rules of Interpretation to choose among them and say why. The obvious way to improve that is to improve the retrieval. Better recall means the correct code is present in the candidate set more often, which means the reranker gets a shot at questions it previously had no chance on. The correct answer can't be selected if it was never retrieved. Straightforward. So we tried. Dense embeddings over the ruling text. Hybrid sparse-plus-dense with score fusion. Larger candidate sets, on the reasoning that if the reranker is any good, giving it more to work with can only help. Recall went up. Final accuracy went down. Not dramatically, and not on every configuration, but consistently and in the wrong direction. Which is worse than a big obvious failure, because it took a while to believe. ## What was happening Once we stopped being irritated and started looking at the individual cases, the mechanism was clear enough. Raising recall doesn't only add the correct code to the candidate set. It adds more plausible wrong codes. Near-neighbours that are semantically close, share a chapter, describe something genuinely adjacent. That's exactly what better retrieval is for — it finds things that resemble the query — and in this domain the things that resemble the query are the distractors. In tariff classification, the codes that look most like the right answer are precisely the ones that are hardest to rule out. That's not incidental, it's structural. The schedule is a hierarchy built by repeatedly subdividing, so the sibling of the correct subheading is by construction the most confusable option available. A candidate set of eight tightly clustered near-misses is a harder discrimination problem than a set of four looser ones, even when the correct code is present in both. We were handing the reranker more chances to be right and more ways to be wrong, and the second effect turned out to be larger. There was a smaller compounding effect too. Bigger candidate sets meant longer context, and the notes and reasoning that actually decide the case got diluted among candidate material that didn't matter. The signal-to-noise ratio in the reranker's input dropped as recall rose. ## The actual bottleneck Reranker discrimination, not retrieval recall. The limiting factor is the system's ability to apply the GRI to distinguish between candidates that are already close. To read a chapter note and determine that it excludes one of them. To judge essential character under GRI 3(b) between a steel body and a plastic housing, and to have a defensible reason for the call. That is not a search problem. It's the legal reasoning problem, and it's the genuinely hard part of the task — the part a human expert spends their time on. Improving the search was optimising the step that was already working well enough. Worth being precise about the shape of the finding: recall was not the constraint past a certain point. Below that point it certainly was, and the early retrieval work was the single biggest improvement we made. There's a threshold, and we spent several weeks on the far side of it without noticing we'd crossed it. ## Why this is worth publishing Because it's the opposite of what the field's instincts say, and because "improve the retrieval" is the first place anyone building one of these systems will reach. The RAG playbook is well established and mostly correct. Recall is the constraint. Better embeddings help. Hybrid search helps. More candidates help. Those hold across a lot of tasks and they held for us at first. They stopped holding on this task, on this corpus, past a certain point. We can't tell you the finding generalises — it's one task, one harness, one corpus, one family of models — and anyone claiming a universal law from a single ablation set is doing the thing we complained about in an earlier post. But if you're building retrieval over a dense hierarchical taxonomy where the distractors are near-neighbours by construction — classification schemes, medical coding, legal citation, product catalogues — we'd suggest measuring the end-to-end number before assuming that recall improvements convert. The specific trap is that recall is easy to measure and end-to-end accuracy is expensive to measure. So teams optimise the cheap metric and report progress on it, and the progress is real, and it doesn't arrive at the output. ## What it changed We stopped spending on retrieval and started spending on the rerank step. How the GRI is actually applied and in what order. What material the model is given to reason over, and specifically whether the relevant section and chapter notes are present in context at the moment they matter rather than a few thousand tokens earlier. Whether the model is asked to eliminate candidates explicitly or just to pick one. And the one with the most operational value: when the system should decline to answer. That last item is where a negative result turns useful. If discrimination between close candidates is the bottleneck, then the cases where the system is genuinely uncertain are identifiable rather than random. They're the ones where the retrieved candidates cluster tightly and the rerank scores sit close together — which is exactly the signal we'd been treating as a good outcome. So the pipeline can flag them. And for a customs broker carrying the liability, "these four are too close to call, here's the ruling behind each and here's the note that would decide it" is a considerably more useful output than a confident wrong code with a high score attached. It also changes the economics of the human review budget. If uncertain cases are identifiable, review effort goes to the cases that need it instead of a fixed sample of everything, and the same number of reviewer hours catches substantially more. We'd rather ship a system that knows the edge of its own competence than one that's two points better on a benchmark and silent about where it fails. Methodology and the full set of ablations are in the [HTS classification report](https://freight.rsaxb.com/report-hts-classification). --- Blog index: https://www.rsaxb.com/blog Site context for agents: https://www.rsaxb.com/llms.txt