loading
loading
A test for whether a model can find one specific fact buried deep inside a huge context.
You plant a unique sentence (the needle) somewhere in a massive document (the haystack) and ask the model to retrieve it, measuring accuracy as a function of context length and position. It matters because long-context models often degrade in the middle of the window, so this test reveals whether their advertised window is actually usable. Example: hiding "the secret code is 7492" in the middle of a 200k-token transcript and checking if the model can quote it back exactly.
Plainly
Think of Needle in a Haystack as a simple recipe for doing the work better. A test for whether a model can find one specific fact buried deep inside a huge context.
In practice
Use it when you need a repeatable method instead of guessing from vibes. In practice, define the owner, input, output, and failure mode before you rely on it.