AI_SLANG_ENTRY
Test-Time Compute Meaning
The extra thinking budget an AI system spends after the prompt arrives: more reasoning steps, search, sampling, verification, or tool use before it commits to an answer.
What does Test-Time Compute mean?
The extra thinking budget an AI system spends after the prompt arrives: more reasoning steps, search, sampling, verification, or tool use before it commits to an answer.
Test-time compute is the work a model or AI system does while answering, not while training. More budget can mean longer reasoning, multiple candidate answers, verification passes, or tool calls before the final response.
Origin and usage
Moved from inference and reasoning research into builder vocabulary as teams learned that scaling compute during inference can improve hard-task performance without changing the model weights.
Source type: paper. Last checked: 2026-07-27.
Research-backed inference-scaling term. The Snell et al. paper is a clean reference for the idea that extra compute at test time can be traded against model size on some tasks; current product usage is broader and provider-specific.
Why test-time compute matters
Training compute makes the model. Test-time compute is spent later, each time the model handles a task. That distinction matters because a system can sometimes improve reliability by working longer on the same prompt instead of requiring a larger model.
For product builders, the trade is visible in latency and cost. More reasoning budget may improve math, coding, planning, or verification, but it can also slow the response and burn more tokens or tool calls.
Common forms
- Generating several candidate answers and choosing the best one.
- Running a verifier, judge, or critic over a draft.
- Letting a reasoning model spend more steps before answering.
- Searching, calling tools, or retrieving evidence before final output.
Examples
- The small model looked weak until we gave it more test-time compute and a verifier.
- Reasoning models are basically selling test-time compute as a product knob.
FAQ
What is test-time compute?
Test-time compute is the compute an AI system spends during inference, after receiving a prompt, to reason, sample, verify, search, or call tools before returning an answer.
Is test-time compute the same as training compute?
No. Training compute is used to create or fine-tune the model. Test-time compute is spent when the model is already deployed and working on a specific request.
Why do reasoning models use more test-time compute?
Reasoning models often spend additional inference steps on hard tasks, which can improve answers but usually increases latency and cost.