A model's response is considered truncated when it stops generating output prematurely because it has hit the max_tokens limit set in the request. This can result in incomplete sentences or code snippets, and can be a frustrating and quiet failure mode.
How Coolhand detects truncation
Coolhand now automatically detects when a response was truncated. For providers that include a finish_reason in their API response, Coolhand inspects this field. If the reason indicates the output was stopped due to its length (for example, a finish_reason of length for OpenAI-compatible models), Coolhand records that the response was truncated.
This detection is only possible for providers that supply a finish reason. Truncation is not currently detected for the following providers:
- Copilot
- ElevenLabs
- TypeSafe
- Embeddings providers
How Coolhand uses this information
Currently, this truncation data is used internally by Coolhand's own systems. For example, the cost-optimization agent can use this signal to recommend increasing your max_tokens request parameter if it observes that a prompt is frequently resulting in truncated outputs. This can help you improve response quality without manual investigation.
The max_tokens you request is different from the model's own absolute maximum output token limit. You can find the model's limit in the Coolhand model catalog.
Checking for truncation in your logs
The truncation signal is stored internally on the LLM request log. However, this information is not yet exposed in the v2 REST API, so you cannot directly query for logs that were truncated. We plan to expose this in a future release.
For a complete reference on the data available via the API, see What every field on an LLM request log means.