Module 3 · Lesson 3 of 4
Three ways a typed call ends badly
- What happenedA typed call that was cut off raised a validation error and returned nothing, a reply of the wrong shape raised too, and a reply with no text block returned None without raising.
- Why it changedThe SDK validates the reply text against your type after the call, so broken or mismatched JSON fails loudly, but when there is no text to validate it has nothing to reject and hands back None.
- How it is measuredOne real call is forced to truncate with a tiny max_tokens and its exception is captured, the same request is sent untyped to see the raw bytes, and the parser is driven locally with hand-made replies.
- How to improve itCatch the validation error and check for None as two separate branches, size max_tokens for the whole reply, and write every outcome, including a raise, into the run file.
Where each reply ends up
Try it. Choose one of the six recorded replies, including the current model's cut-off one, and see which of the three exits it takes, and which guard would catch it.
Where this data comes from · 2026-10-14, 2026-11-20 and 2026-09-02
Read from the book's saved record. Nothing here is simulated.
runs/ch3-router-typed-2026-10-14.txt section Cruns/ch3-course-route-typed-2026-11-20.jsonlruns/ch7-parse-output-format-2026-09-02.txt sections C3 and C4
The same cut-off reply, typed and untyped
Try it. Switch between what your code receives and what the model wrote, for the same cut-off request sent typed and untyped.
Where this data comes from · claude-opus-5 · 2026-09-02
Read from the book's saved record. Nothing here is simulated.
runs/ch7-parse-output-format-2026-09-02.txt sections C1 and C1b
What happened
The guarantee from Lesson 1 was "one of four values, or an exception". This lesson tests the second half on purpose, and finds a third outcome the slogan leaves out.
Cut off: it raised. The typed router was called once with max_tokens=3, too few to finish the JSON. Nothing was returned. The call raised a Pydantic ValidationError with this message (runs/ch3-router-typed-2026-10-14.txt, section C):
Invalid JSON: EOF while parsing a string at line 1 column 10
The reply had got as far as {"queue":" and stopped. On the course run on claude-opus-5, the same call cut off even earlier and raised the same exception class, with Invalid JSON: EOF while parsing a string at line 1 column 5 on the fragment {"que (runs/ch3-course-route-typed-2026-11-20.txt). This time the outcome is a row in the run file.
Wrong shape: it raised. Driving the parser locally with valid JSON that used the wrong field name raised as well, reporting 2 validation errors, one for each required field that was missing. A reply that was a plain sentence raised too (runs/ch7-parse-output-format-2026-09-02.txt, section C3).
No text at all: it returned None. Driving the parser with a response that had no content blocks produced no exception. The recorded line is parsed_output = None -- no raise (same file, section C4).
| Reply | Outcome | Caught by except ValidationError |
Caught by a None check |
|---|---|---|---|
| Complete and valid | your type | not needed | not needed |
| Cut off mid-JSON | raises | yes | no |
| Valid JSON, wrong shape | raises | yes | no |
| Not JSON | raises | yes | no |
| No text block | None, no error |
no | yes |
Why it changed
The check happens after the call, on your machine. The SDK takes the reply's text and asks Pydantic to build your type from it. Half a JSON object cannot be built into anything, so Pydantic refuses. So does JSON with the wrong fields.
The third case slips through for a dull reason: with no text block there is no text to validate. Nothing is malformed. The SDK returns the message with its parsed field empty.
Now compare the same cut-off reply through a typed and an untyped call, using the second comparison above. On an invoice schema with max_tokens=16, the typed call raised Invalid JSON: EOF while parsing a value at line 1 column 30. The same request sent untyped came back as an ordinary response: stop reason max_tokens, 16 output tokens, and a string holding the first part of a JSON object (runs/ch7-parse-output-format-2026-09-02.txt, sections C1 and C1b, claude-opus-5). These were two calls with the same small budget, and both replies stopped partway through the object. Only one path let its fragment reach your code looking like a result.
This connects to Module 1. There, a model that thinks before answering spent a small budget entirely on thinking, and 20 of 20 answers came back with no text (BOOK.md, Measured findings). A reply with no text is exactly the input that produces None in the local probe. On the one course run so far, a typed call on the thinking model with three tokens of budget raised instead of returning None. One call is not a rule, which is why the hands-on test asks you to predict and record what yours does.
How it is measured
- Force the failure. One real call with a budget too small to finish. The exception class, its message and the fragment are captured from the console.
- Look at the bytes. The same request through the untyped path shows what the typed path refused.
- Drive the parser directly. Four hand-made replies and one empty one are fed to the SDK's parsing function with no network involved. These are local facts about the SDK version named in the file, SDK 1.2.0, not observations of a model.
- Notice what was not recorded. The truncation result exists only in the console capture. The call raised before the harness reached the code that saves usage and writes the file, so the run's JSONL has no row for it. The chapter calls this a defect in the instrument and leaves it on the record.
How to improve it
- Two guards, not one. Wrap the call in
except ValidationError, and separately check the parsed result forNonebefore using it. - Decide what each means. A raise from truncation may deserve one retry with a larger budget. A
Nonedeserves a look at the stop reason first. - Size
max_tokensfor the whole reply, and on a model that thinks, for the thinking as well. - Record the failure where the data lives. Write the outcome of every call to the run file inside the
exceptbranch, so a raise leaves a row. The hands-on test checks for it. - Never loosen the parser to make an error go away. A parser that accepts half an object is how a truncated key lands in a database.
The edge cases are set out in Chapter 3 of the book, "Edge cases". Next: typed is not correct.
Every number in this lesson, and the file it came from (7)
| Value | File in the book's record | Conditions |
|---|---|---|
| Invalid JSON: EOF while parsing a string at line 1 column 10 | runs/ch3-router-typed-2026-10-14.txt | claude-opus-4-6, max_tokens=3, one call |
| Invalid JSON: EOF while parsing a string at line 1 column 5 | runs/ch3-course-route-typed-2026-11-20.txt | claude-opus-5, max_tokens=3, one call |
| 2 | runs/ch7-parse-output-format-2026-09-02.txt | |
| parsed_output = None -- no raise | runs/ch7-parse-output-format-2026-09-02.txt | |
| Invalid JSON: EOF while parsing a value at line 1 column 30 | runs/ch7-parse-output-format-2026-09-02.txt | claude-opus-5, max_tokens=16, 2026-09-02, one call |
| 16 | runs/ch7-parse-output-format-2026-09-02.txt | |
| 20 of 20 | BOOK.md | claude-opus-5, max_tokens 100, 2026-11-08 |
Reference: Chapter 3, "Edge cases" and the architecture diagram description.