Measured Academy Measured Academy
Français Pilot Progress 0

Module 4 · Lesson 4 of 4

Where the loop breaks

  1. What happenedReading only the first content block drops a second tool request, in the worst case without any error, answering one of two requests was rejected by the API, a failed tool did not stop the model, and the same six prompts run twice gave different traces.
  2. Why it changedA response is a list of blocks whose first item is not guaranteed to be the one you want, the API requires a result for every request, and the sequence of steps is sampled just as the text of an answer is.
  3. How it is measuredRequests are sent with complete, reordered and partial results to see which the API accepts, a failing result is returned to see the stop reason, and the same prompts are run twice and the traces compared.
  4. How to improve itIterate every block, answer every request, put the decision about a failed tool in your own code, treat tool results as untrusted input, and never assert an exact step count in a test.
Side by side

Answer all of them, in any order

Try it. Build the turn you send back: include or leave out each of the two results, and reverse their order.

Where this data comes from · SDK 1.2.0 · 2026-08-27

Read from the book's saved record. Nothing here is simulated.

  • BOOK.md Measured findings, rows "Two tool_results in one user turn" and "Partial tool results"
Switch

The same prompts, run twice

Try it. Switch between the first and second run, in the Chapter 4 record and then in the Chapter 8 record.

Where this data comes from · claude-opus-4-6 · 2026-08-28 and 2026-09-02

Read from the book's saved record. Nothing here is simulated.

  • BOOK.md Measured findings, row "Trace-level nondeterminism"
  • BOOK.md errata, the entry that removed that row's caveat

The code was identical between runs in both records. BOOK.md does not say which of the six prompts these were.

What happened

Five ways this loop goes wrong, four of them on the record in BOOK.md, Measured findings.

The first block is not the whole response. A question with two separate sums in it returned 2 tool request blocks in one response. A loop that reads only the first block runs one tool and ignores the other. If it also sends back only that block, the model goes on to answer from a result set with a hole in it, and nothing errors; that is the case the chapter describes. If it sends the whole response back with one result, the next point applies instead.

Partial results are rejected. Return both results and the API accepts the turn. Reverse their order and it still does. Return one of two and the request fails with status 400 and a message that tool request ids were found without a matching result.

A failed tool does not stop the model. A result marked as an error, carrying a division-by-zero message, was accepted. The next response's stop reason was end_turn: the model acknowledged the failure and answered by another route. One run, one prompt.

A thinking block can come first. On a model that thinks by default, the first content block was a thinking block with no text attribute (claude-opus-5, 2026-09-07). It is the same defect as the dropped tool call, arriving from a model change instead of a change in the question.

The trace itself moves. The same six prompts were run twice in a row. One of them took 1 step and 0 tool calls on the first run, then 3 steps and 4 tool calls on the second. The course's three attempts on claude-opus-5 show it again: the mice question took three steps, then four, then three (2, 1, 3, 2, 3, 1; 2, 1, 3, 2, 4, 1; 2, 1, 3, 2, 3, 1), and in the second attempt it took three again when re-run moments later. That last repeat is what broke the course's first forced-run design (Lesson 2).

Why it changed

The first four share a cause. A response is a list of blocks, and the API holds you to a contract about that list:

You do The API does Why that is the better design
Answer every request, any order accepts order carries no meaning; ids do the matching
Answer some requests rejects the turn a dropped block becomes an error, not a degraded answer
Read block zero and stop nothing, it cannot see your loop this one is yours to prevent

Explore the combinations in the first viewer above. The rejection is the helpful case. The dangerous one is the bug that never reaches the API: your loop silently builds a complete-looking turn from half the requests.

The fifth has a different cause, and it is Module 1's lesson one level up. There, the wording of an answer varied under a fixed prompt. Here the sequence of steps varies. Nothing in the code changed between the two runs.

The first record of this was taken before the lookup tool was fixed to accept product names, which left room to blame the broken tool. That caveat was later removed: Chapter 8 reproduced the effect on identical code with a working tool, where 2 of 6 prompts changed step count between consecutive runs (BOOK.md, errata).

How it is measured

  • Probe the contract. Send the same two results complete, reversed and with one missing, and record what comes back each time.
  • Return a failure on purpose and log the stop reason of the next response.
  • Run the same prompts twice and compare traces line by line. A difference with no code change is the finding.
  • Know what stands behind the table. The book's original output was not saved. What supports it now is a later harness, built for Chapter 10 from this chapter's printed code, which reproduced the step table on 18 of 18 prompt runs across three arms (BOOK.md). The course's runs on a current model are kept whole, every request, result and answer, in runs/. The table reproduced, and the trace also moved, in Chapter 8 and in the course's attempts. Both are true: a result can be typical and still be a sample.

How to improve it

  • Iterate every block and filter by type. Pick out tool requests by their type, and text by its type. This one habit covers the second request and the thinking block.
  • Build one result per request, always. If a tool raises, return an error result for that id; do not skip it.
  • Decide what failure means in your code. The model will carry on after an error result. If the request should stop, stop it above the loop.
  • Do not assert a step count in a test. Assert a bound, or assert the outcome. A cost estimate from one observed trace rests on one draw.
  • Treat tool results as untrusted input. Whatever a tool returns becomes model input in the middle of the process. Module 11 is the full treatment.
  • Know when not to build a loop. The chapter lists four cases: the steps are known in advance, there is only one tool, there is a hard limit on latency, or you cannot authorise what the tool would do.

The edge cases are in Chapter 4 of the book, "Edge cases". Now take the hands-on test.

Every number in this lesson, and the file it came from (9)
ValueFile in the book's recordConditions
2BOOK.md
400BOOK.md
1 step and 0 tool callsBOOK.mdclaude-opus-4-6, SDK 1.2.0, 2026-08-28, two consecutive runs
3 steps and 4 tool callsBOOK.md
2, 1, 3, 2, 3, 1runs/ch4-course-agent-2026-11-20.txtclaude-opus-5, 2026-11-20, first attempt, max_tokens 1024
2, 1, 3, 2, 4, 1runs/ch4-course-agent-2026-11-20-run2.txtclaude-opus-5, 2026-11-20, second attempt
2, 1, 3, 2, 3, 1runs/ch4-course-check-2026-11-20-run3.txtclaude-opus-5, 2026-11-20, third attempt, check PASS
2 of 6BOOK.md
18 of 18BOOK.mdChapter 10's harness, built from Chapter 4's printed code, across three arms

Reference: Chapter 4, "Edge cases" and "When NOT to use this".