Module 2 · Lesson 2 of 4
The chain that lost to one call
- What happenedA three-call routing chain matched the answer key on fewer tickets than a single call, while making three times as many calls.
- Why it changedEach extra call passed free text to the next one, and the final step returned queue names wrapped in markdown that an exact comparison rejects.
- How it is measuredBoth versions run over the same tickets against a key written before the run, scored by exact match, with calls and tokens logged per stage.
- How to improve itStart with one call that names each option and asks for one word; add a step only when a measured failure calls for it.
Ticket by ticket, chain against single call
Try it. Pick a ticket, or look at all ten, and switch the scoring between exact match and letters only.
Where this data comes from · claude-opus-4-6, 2026-09-07; claude-opus-5, 2026-11-20
Read from the book's saved record. Nothing here is simulated.
runs/ch2-collapse-2026-09-07.jsonlruns/ch2-collapse-2026-09-07.txtruns/ch2-course-collapse-2026-11-20.jsonlruns/ch2-course-collapse-2026-11-20.txt
What happened
The same 10 support tickets were routed two ways, to one of four queues, on claude-opus-4-6 on 2026-09-07.
The chain made three calls per ticket: summarise the ticket, list the queues that could own it, pick one. The single call made one: here are the four queues and what each owns, reply with one word.
| Chain, book run | Single call, book run | Chain, course run | Single call, course run | |
|---|---|---|---|---|
| Model | claude-opus-4-6 |
claude-opus-4-6 |
claude-opus-5 |
claude-opus-5 |
| Calls | 30 | 10 | 30 | 10 |
| Exact match with the key | 4 of 10 | 10 of 10 | 8 of 10 | 10 of 10 |
| Returns that were not a queue name | 6 of 10 | 0 of 10 | 1 of 10 | 0 of 10 |
Sources: runs/ch2-collapse-2026-09-07.txt for the book's run, and runs/ch2-course-collapse-2026-11-20.txt for the course's run of the same exercise on a current model, made on 2026-11-20 with the course harness. On both models the design with more steps did worse on the score and made three times the calls. Use the "Recorded run" switch above the comparison to move between the two.
BOOK.md records two trials that day, with the chain at 5 of 10 on the other one. Only the trial above has a file in runs/, so it is the one this lesson shows. The chain's score moved between trials; the single call's did not.
Why it changed
Step through one ticket in the comparison above and look at what the chain's last call returned. On most of the tickets it missed, the queue was right and the format was not: the name came back in bold, with asterisks around it.
That happened because of what each call was handed. The single call got the ticket and a tight instruction. The chain's last call got a paragraph of the model's own prose listing candidate queues, formatted the way models like to format lists, and echoed that style back.
Where the tokens went tells the same story, more loudly on the newer model:
| Stage, output tokens over 10 calls | Book run, claude-opus-4-6 |
Course run, claude-opus-5 |
|---|---|---|
| Chain, summarise | 219 | 1,753 |
| Chain, list candidates | 488 | 6,969 |
| Chain, pick | 52 | 141 |
| Single call | 40 | 195 |
The middle step wrote the most text on both models, and all of it became input for the step that then had to produce one word. On the current model, which thinks before it answers, 3,369 of the run's output tokens were thinking (runs/ch2-course-collapse-2026-11-20.txt).
What changed between the models is the format, not the shape. The newer model put a queue name in decoration only once, so the chain's exact score rose. It still lost to the single call, and the ticket it got wrong was a decision, not a format: Lesson 3 takes that apart.
How it is measured
Four things make this comparison fair, and you will reproduce each one:
- Same tickets for both versions.
- A key written first. The answer key is a judgement, one queue per ticket with a reason, written before the single call was ever run. Yours may differ on a ticket or two, and that is expected.
- Exact match. The return, stripped and lowercased, equals the key or it does not. This is what a caller using the return as a dictionary key would experience.
- Usage per call. Every call logs its stage, input tokens and output tokens.
| Whole run | Book run, claude-opus-4-6 |
Course run, claude-opus-5 |
|---|---|---|
| Calls | 40 | 40 |
| Input tokens | 2,599 | 8,142 |
| Output tokens | 799 | 9,058 |
| Cost | $0.03 (COST.md) |
$0.27 (BOOK.md, Measured findings) |
The same forty calls cost several times more on the newer model, almost all of it output the chain's middle step wrote.
One sample of each version is not a rate. The lesson reports what one recorded trial on each model did.
How to improve it
- Write the single call first. Name each option, say what it owns, ask for one word. Measure it before adding anything.
- Add a step only for a measured reason. "It might help to summarise first" is a guess. A ticket the single call gets wrong, twice, is a reason.
- If you keep a chain, tighten the hand-offs. Each intermediate call should return the smallest thing the next one needs.
- Constrain the final shape. Module 3 replaces "reply with one word" with a typed return that cannot be a queue name in bold.
The chain and the tickets are printed in Chapter 2 of the book, "Hands-on exercise". Next: wrong answer or wrong format.
Every number in this lesson, and the file it came from (28)
| Value | File in the book's record | Conditions |
|---|---|---|
| 10 | runs/ch2-collapse-2026-09-07.txt | |
| 30 | runs/ch2-collapse-2026-09-07.txt | |
| 10 | runs/ch2-collapse-2026-09-07.txt | |
| 4 of 10 | runs/ch2-collapse-2026-09-07.txt | claude-opus-4-6, 2026-09-07, the author's key |
| 10 of 10 | runs/ch2-collapse-2026-09-07.txt | claude-opus-4-6, 2026-09-07, the author's key |
| 8 of 10 | runs/ch2-course-collapse-2026-11-20.txt | claude-opus-5, 2026-11-20, one run, predictions by an agent working for the author |
| 10 of 10 | runs/ch2-course-collapse-2026-11-20.txt | claude-opus-5, 2026-11-20, one run, predictions by an agent working for the author |
| 6 of 10 | runs/ch2-collapse-2026-09-07.txt | |
| 0 of 10 | runs/ch2-collapse-2026-09-07.txt | |
| 1 of 10 | runs/ch2-course-collapse-2026-11-20.txt | |
| 0 of 10 | runs/ch2-course-collapse-2026-11-20.txt | |
| 5 of 10 | BOOK.md | the other of two trials on 2026-09-07; only one trial has a file in runs/ |
| 219 | runs/ch2-collapse-2026-09-07.txt | |
| 1,753 | runs/ch2-course-collapse-2026-11-20.txt | |
| 488 | runs/ch2-collapse-2026-09-07.txt | |
| 6,969 | runs/ch2-course-collapse-2026-11-20.txt | |
| 52 | runs/ch2-collapse-2026-09-07.txt | |
| 141 | runs/ch2-course-collapse-2026-11-20.txt | |
| 40 | runs/ch2-collapse-2026-09-07.txt | |
| 195 | runs/ch2-course-collapse-2026-11-20.txt | |
| 3,369 | runs/ch2-course-collapse-2026-11-20.txt | |
| 40 | runs/ch2-collapse-2026-09-07.txt | |
| 2,599 | runs/ch2-collapse-2026-09-07.txt | |
| 8,142 | runs/ch2-course-collapse-2026-11-20.txt | |
| 799 | runs/ch2-collapse-2026-09-07.txt | |
| 9,058 | runs/ch2-course-collapse-2026-11-20.txt | |
| $0.03 | COST.md | claude-opus-4-6, Opus rates in COST.md |
| $0.27 | BOOK.md | claude-opus-5, 2026-11-20, Opus rates in COST.md |
Reference: Chapter 2, "Before / After" and "Hands-on exercise - collapse the chain".