loading
loading
Sending model output token-by-token as it's generated instead of waiting for completion.
Streaming dramatically improves perceived latency because users see words appear immediately rather than waiting for the full response. Most LLM APIs expose a streaming mode via server-sent events or WebSockets. Building streaming UIs requires handling partial JSON, managing render state, and gracefully handling mid-stream errors.
Plainly
Think of Streaming as a tool from the builder toolbox. Sending model output token-by-token as it's generated instead of waiting for completion.
In practice
Use it when you are installing, editing, testing, reviewing, or shipping code. In practice, define the owner, input, output, and failure mode before you rely on it.