# Detecting Truncated LLM Output: finish_reason, stop_reason and the max_tokens Tail

This is the shape of the line most tutorials use to read a model's answer:

```python
text = response.content[0].text
```

It reads the answer. It never asks whether the answer is finished. And the responses where that question matters most are not a random sample of your traffic. They're your biggest and hardest inputs.

This post covers what the stop signal is on each major API, why `max_tokens` truncation concentrates in the tail of your input distribution, the three layers that hide it, and a small gate that makes it impossible to miss.

## The field next to the text

Every major model API returns a stop signal alongside the generated text. They're named differently, but they map to the same outcomes:

| Outcome | OpenAI Chat Completions | OpenAI Responses | Anthropic Messages | Gemini |
|---|---|---|---|---|
| Finished naturally | `finish_reason: "stop"` | `status: "completed"` | `stop_reason: "end_turn"` | `finishReason: STOP` |
| Hit the output cap | `"length"` | `status: "incomplete"`, reason `max_output_tokens` | `"max_tokens"` | `MAX_TOKENS` |
| Wants a tool | `"tool_calls"` | function call items in `output` | `"tool_use"` | function call parts |
| Blocked or refused | `"content_filter"` | refusal content | `"refusal"` | `SAFETY`, `RECITATION`, `PROHIBITED_CONTENT`, … |
| Filled the context window | — | — | `"model_context_window_exceeded"` | — |

Two details matter more than the table. First, only the "finished naturally" row means the model completed its answer. Second, none of these rows is an HTTP error. Anthropic's docs state that refusals come back as a normal 200. A cut-off response is a 200 too.

## What a cut looks like, by output shape

`max_tokens` is a meter outside the model, not an instruction inside the prompt. When the count hits the limit, generation stops wherever it is. The damage depends on what the model was writing:

![Loud cuts vs silent cuts: prose, strict JSON and code fail visibly; lists, repaired JSON, summaries, empty reasoning replies and tool calls fail silently.](https://cdn.hashnode.com/uploads/covers/6a44b5d24b41ab0145e5cf63/186fe409-aa1e-4dc6-b779-d79f5026c640.png)

- **Prose**: stops mid-sentence. Loud.
- **Strict JSON**: fails to parse. Loud, unless something repairs it.
- **Lists and line-delimited records**: lose their tail. Every item before the cut is complete, so dropping the partial last line produces something that looks finished. Silent.
- **Summaries**: cover the first part of the document and never mention the rest. Silent.
- **Tool calls**: Anthropic documents the case where a `max_tokens` response ends with an incomplete `tool_use` block. The action never runs. Silent, and the most damaging. (Background on how these calls work: [tool calling and MCP explained](https://simplyexplained.hashnode.dev/how-ai-learned-to-actually-do-things-tool-calling-and-mcp-explained).)
- **Streams**: Anthropic delivers `stop_reason` in the final `message_delta` event. A handler that saves on stream close never sees it.

## Why truncation concentrates in the tail

If length stops were spread evenly, a 0.5% rate would be a rounding error. They aren't spread evenly. Two forces push them toward the requests that matter most.

**Output size tracks input size.** A 200-line invoice needs roughly twenty times the extraction output of a 10-line one. A translation is about as long as its source. Your `max_tokens` is one number; the output your inputs need is a long-tailed distribution.

**Reasoning spends the same budget first.** OpenAI's reasoning guide says reasoning tokens are billed as output tokens, occupy context, and can range from a few hundred to tens of thousands depending on problem complexity. Anthropic's extended thinking docs say thinking tokens count toward `max_tokens`, and in manual mode `budget_tokens` must be less than `max_tokens`. Harder problems think longer and leave less room for the answer. (When that thinking helps at all is its own question: [when to use reasoning models](https://simplyexplained.hashnode.dev/when-to-use-reasoning-models).)

![One budget, two spenders: reasoning and answer share the max_tokens cap; the hardest input spends most of it thinking and its answer is cut.](https://cdn.hashnode.com/uploads/covers/6a44b5d24b41ab0145e5cf63/b8970c9f-82e9-4923-9cb2-c174bc129d71.png)

The extreme case is documented: OpenAI notes a response can hit the limit before any visible output is produced. Empty answer, billed reasoning.

The consequence for testing is the important part. Eval sets are usually sampled from typical inputs, and typical inputs fit. **Your evals pass for the same reason production fails.**

## Three layers that hide it

1. **The status code.** 200 means delivered.
2. **The repair layer.** Libraries like `json_repair` say they close missing brackets, fix truncated values and fill missing values with defaults so the output is valid. Useful on a finished answer with a stray comma. On a cut answer, it converts a failure into a shorter valid result. A schema guarantees shape, not truth, and not completeness either ([structured outputs still hallucinate](https://simplyexplained.hashnode.dev/structured-outputs-still-hallucinate-designing-llm-schemas-that-can-say-i-don-t-know)).
3. **The success metric.** Error-rate dashboards count exceptions. A length stop doesn't raise one.

## The gate: read the reason before the text

Make the stop reason a required branch, and make unknown values fail closed. Stop-reason enums grow over time.

**Anthropic Messages API**

```python
class Truncated(Exception): ...
class Refused(Exception): ...
class NeedsToolRun(Exception): ...
class UnexpectedStop(Exception): ...

def finished_text(resp):
    reason = resp.stop_reason
    if reason in ("end_turn", "stop_sequence"):
        return "".join(b.text for b in resp.content if b.type == "text")
    if reason == "tool_use":
        raise NeedsToolRun(resp)
    if reason in ("max_tokens", "model_context_window_exceeded"):
        raise Truncated(reason, resp.usage.output_tokens)
    if reason == "refusal":
        raise Refused(resp.stop_details)
    raise UnexpectedStop(reason)  # pause_turn, and anything added later
```

**OpenAI Responses API**

```python
def finished_text(resp):
    if resp.status == "incomplete":
        raise Truncated(resp.incomplete_details.reason)  # e.g. "max_output_tokens"
    if resp.status != "completed":
        raise UnexpectedStop(resp.status)
    return resp.output_text
```

**OpenAI Chat Completions**

```python
choice = resp.choices[0]
if choice.finish_reason == "length":
    raise Truncated("length", resp.usage.completion_tokens)
if choice.finish_reason not in ("stop", "tool_calls"):
    raise UnexpectedStop(choice.finish_reason)
```

**Gemini (google-genai)**

```python
from google.genai import types

cand = resp.candidates[0]
if cand.finish_reason == types.FinishReason.MAX_TOKENS:
    raise Truncated("MAX_TOKENS")
if cand.finish_reason != types.FinishReason.STOP:
    raise UnexpectedStop(cand.finish_reason)
```

Put any repair step **after** this gate. Repair is for syntax noise on finished output, never for a length stop.

## Sizing: raise the cap, then stop relying on it

Raising `max_tokens` is usually the right first move after a switch to reasoning models; OpenAI recommends reserving at least 25,000 tokens for reasoning and output when you start. But it moves the tail rather than removing it: every model has an output ceiling, input and output share the context window, and the cap doubles as your runaway-cost circuit breaker.

What actually works:

- **Measure the tail.** Log output tokens (reasoning broken out where reported) per request, bucketed by input size. Size the cap from the biggest bucket plus headroom, and alert on length-stop rate per bucket.
- **Bound the output, not the input.** Split by how much the model must *write*: extract lines 1–50, then 51–100; one section per summary call. More calls re-read the same context, so keep the shared prefix cache-friendly ([prompt caching for AI agents](https://simplyexplained.hashnode.dev/trimming-prompt-raises-ai-bill)).
- **Prefer retry or split over continuation for structured output.** Continuing a cut JSON array risks duplicated or skipped items at the seam, and a response cut during reasoning has no visible partial to continue. For a cut tool call, Anthropic's guidance is to retry with a higher `max_tokens`.

## Leave a finish line, then test the tail

The stop reason is the primary signal. A finish line catches the layers that drop it (proxies, streaming handlers, text-only wrappers):

- a required **last** field in the JSON schema, which a cut response can't contain;
- a fixed end marker for free text;
- a **count check** against an independent source, such as your own parser's line count for the invoice.

Then add the inputs your random sample will never include:

```python
import pytest

BIGGEST = load_cases("largest_inputs.jsonl")        # top 10 by size
HARDEST = load_cases("most_reasoning_tokens.jsonl")  # top 10 by reasoning usage

@pytest.mark.parametrize("case", BIGGEST + HARDEST, ids=lambda c: c["id"])
def test_tail_inputs_finish(case):
    resp = call_model(case["input"])
    assert stop_reason(resp) in NATURAL_STOPS
    items = parse_items(resp)
    assert len(items) == case["expected_count"]
```

`load_cases`, `call_model`, `stop_reason` and `parse_items` are your own helpers; the point is that the suite is built from the tail, not sampled from the middle.

![The Finish Line Check: read the reason before the text, never repair what was cut, budget for the tail, bound the output, leave a finish line.](https://cdn.hashnode.com/uploads/covers/6a44b5d24b41ab0145e5cf63/409423d0-72a9-4c1d-b9ff-f28afa4c9575.png)

## Key Takeaways

- A truncated LLM response returns HTTP 200. The stop reason (`finish_reason`, `stop_reason`, `status`, `finishReason`) is the only built-in signal.
- Truncation isn't random. Output grows with input, and reasoning tokens spend the same `max_tokens` budget first, so the cut concentrates on your largest and hardest requests.
- Lists, repaired JSON, summaries, empty reasoning replies and cut tool calls all look finished.
- Gate on the stop reason before reading text; fail closed on unknown values; repair only after the gate.
- Size caps from the tail, split work by output size, add a finish line, and test with your biggest and hardest real inputs.

## FAQ

### What does finish_reason "length" mean in the OpenAI API?
It means generation stopped because the response reached the output token limit you set, not because the model finished. The text you received is partial. On the Responses API the equivalent is `status: "incomplete"` with `incomplete_details.reason` set to `max_output_tokens`.

### Does a truncated LLM response return an error?
No. A response cut off by the token limit returns HTTP 200 like any other. Anthropic's docs note that even refusals arrive as normal 200 responses. You have to check the stop reason field to know whether the answer is complete.

### Do reasoning tokens count against max_tokens?
Yes, on the major APIs. OpenAI bills reasoning tokens as output tokens and counts them against the output limit; a response can run out before any visible text appears. Anthropic's docs say thinking tokens count toward `max_tokens`, so the limit must leave room for the final answer.

### Why does an LLM return an empty response?
With reasoning models, a common cause is that reasoning consumed the entire output budget, leaving nothing for the visible answer; the stop reason will show the limit was hit. Raise the limit or split the task. Anthropic also documents a separate case of near-empty `end_turn` responses after tool results.

### Should I continue a truncated response or retry it?
For prose, continuation (sending the partial back and asking the model to keep going) works reasonably well. For structured output, prefer retrying with a larger limit or splitting the task, since seams can duplicate or skip items. For a tool call cut mid-arguments, Anthropic recommends retrying with a higher `max_tokens`.

### How do I choose a max_tokens value?
Measure real output token usage, including reasoning tokens, grouped by input size, and size the limit from your largest group plus headroom rather than the average. Keep a cap as a cost circuit breaker, and split work so no single call needs an unbounded output.

## Related reading

- [Structured Outputs Still Hallucinate: Designing LLM Schemas That Can Say "I Don't Know"](https://simplyexplained.hashnode.dev/structured-outputs-still-hallucinate-designing-llm-schemas-that-can-say-i-don-t-know)
- [When AI's 'Thinking' Mode Actually Makes It Worse](https://simplyexplained.hashnode.dev/when-to-use-reasoning-models)
- [How AI Learned to Actually Do Things: Tool Calling and MCP Explained](https://simplyexplained.hashnode.dev/how-ai-learned-to-actually-do-things-tool-calling-and-mcp-explained)
- [Prompt Caching for AI Agents: Stop Breaking Your Own Cache](https://simplyexplained.hashnode.dev/trimming-prompt-raises-ai-bill)
- [Offset vs Cursor Pagination: Why Your Pages Skip and Repeat Rows](https://simplyexplained.hashnode.dev/offset-vs-cursor-pagination-skipped-rows): the same "success logged, rows missing" bug, in a database.

## The bottom line

An HTTP 200 from a model API is a delivery receipt. The stop reason is the line that says whether the answer was paid in full, and the requests most likely to come up short are the ones you most needed.

> Don't ask whether the AI answered. Ask why it stopped.

---

New to the series? [Start Here: a plain-English map to understanding AI](https://simplyexplained.hashnode.dev/start-here-understand-ai). Get one of these in your inbox each week: [subscribe on Substack](https://simplyexplainedbyaj.substack.com/subscribe).

*Adam Jaber is a software engineer who writes Simply Explained: complex tech topics, made simple. No jargon, no hype.*