loading
loading
How the model picks the next token from its probability distribution — greedily or with controlled randomness.
After the model assigns probabilities to every next token, sampling decides which one to actually emit. Top-k keeps only the k highest-probability tokens and picks among them; top-p (nucleus) keeps the smallest set of tokens whose probabilities sum to p (e.g. 0.9), so the candidate pool shrinks when the model is confident and grows when it's unsure. These work alongside temperature: tighter sampling means more focused, repetitive output; looser means more creative but riskier. For deterministic, parseable output, sample greedily (top-k=1).
Plainly
Think of Sampling (top-p / top-k) as a simple recipe for doing the work better. How the model picks the next token from its probability distribution — greedily or with controlled randomness.
In practice
Use it when you need a repeatable method instead of guessing from vibes. In practice, define the owner, input, output, and failure mode before you rely on it.