loading
loading
The mechanism that lets each token look at every other token to decide what's relevant.
Attention is the core operation of a transformer: for each token, the model computes how much it should 'attend to' every other token in the context, then mixes their information accordingly. This is what lets a model resolve 'it' back to the right noun three sentences ago, or connect a function call to its definition. The catch is cost — standard attention is quadratic in sequence length, so doubling the context roughly quadruples the compute, which is why long contexts are expensive and why every optimization below exists.
Plainly
Think of Attention as the brain part that guesses or decides. The mechanism that lets each token look at every other token to decide what's relevant.
In practice
Use it when model choice, prompts, latency, cost, or quality affect the product result. In practice, define the owner, input, output, and failure mode before you rely on it.