loading
loading
How long you wait from sending a request until the first token comes back.
TTFT measures the delay before the model starts streaming output, dominated by the prefill phase where the model reads your whole prompt. It's the number users feel as 'is it frozen?' — a 200ms TTFT feels instant, 4s feels broken. Big prompts and cold servers blow it up; prompt caching and shorter context shrink it. Optimize TTFT for chat UX, optimize throughput for batch jobs.
Plainly
Think of Time to First Token (TTFT) as roads and power for the app city. How long you wait from sending a request until the first token comes back.
In practice
Use it when local behavior needs to become a reachable, reliable deployed service. In practice, define the owner, input, output, and failure mode before you rely on it.