Module 4 · Hands-on test
Two tools, a cap that raises, and a trace you can read
You run this on your own machine with your own API key. Nothing you run is sent to this site.
Get the files
Everything this test asks you to run is in one download, laid out exactly as in the companion repository, so every command in the steps works from the unzipped folder.
Download the Module 4 kit (zip, 7 files)
ch04-first-agent/ch03_trace.pych04-first-agent/ch4_course_agent.pych04-first-agent/ch4_course_check.pych04-first-agent/course-agent-2026-11-20-attempt3.txtch04-first-agent/course-agent-2026-11-20-attempt3.jsonlch04-first-agent/course-prediction-2026-11-20-attempt3.jsonch04-first-agent/course-check-2026-11-20-attempt3.txt
Unzip it, open a terminal in module-4-kit, create a Python 3.12 virtual environment, and install what Modules 1 to 4 need:
pip install anthropic==1.2.0 pydantic==2.13.5
Those two pins are read from the kit's requirements.txt, which also lists packages only later modules use. The companion repository is not open to pilot learners yet, so use this download wherever the steps say "companion repo".
Try the check before you spend anything. This runs it on the course's own recorded run and needs no key:
python ch04-first-agent/ch4_course_check.py --trace ch04-first-agent/course-agent-2026-11-20-attempt3.jsonl --prediction ch04-first-agent/course-prediction-2026-11-20-attempt3.json
It should print Module 4 check: PASS. That is the course's result, not yours: do not record it below.
What you will do
Add a calculator as a second tool to the chapter's stock-lookup agent, predict how many steps each of the six questions will take, run them, and read your trace against the chapter's table. Then make the step cap fire on purpose and confirm it raises.
This course is for learners aged 18+.
Cost
You use your own API key, so you pay for this run.
| What | Course run, claude-opus-5 (passing attempt) |
Book run, claude-opus-4-6 |
|---|---|---|
| Model calls | 13: twelve for the six questions, one forced | not recorded |
| Input tokens | 7,730 | not recorded: the run logged no usage |
| Output tokens | 2,430, of which 666 thinking | not recorded |
| Cost of that run | $0.10 | no figure |
| Source | BOOK.md, Measured findings; runs/ch4-course-check-2026-11-20-run3.txt |
COST.md lists Ch 4 among the runs that logged no usage |
The course needed three attempts to pass (Module 4, Lesson 2). The first two cost $0.10 and $0.13. If your run fails its check, a rerun costs about as much again. No single question can make more calls than your cap, and each call sends both tool definitions and the whole message list so far, so later steps cost more input than earlier ones.
Steps
Work from the unzipped kit (the same files as the companion repo's ch04-first-agent/), from its top folder. Your files go in ch04-first-agent/out/. The kit also holds the course's own harness, ch4_course_agent.py, and its passing run. Build yours first; open those afterwards to compare.
- Read the chapter's exercise, then
ch04-first-agent/ch03_trace.py. The script keeps the number it was written under. It is the author's harness and it has two things you will change: when it runs out of steps it falls through instead of raising, and it prints its trace instead of saving it. -
Write your predictions and save them as
ch04-first-agent/out/prediction.jsonbefore any call:{ "predicted_steps": [0, 0, 0, 0, 0, 0], "predicted_at": "2026-01-01T00:00:00+00:00" }One whole number per question, in the order of the script's
PROMPTSlist. Setpredicted_atto the current UTC time. This prints it:python -c "from datetime import datetime, timezone; print(datetime.now(timezone.utc).isoformat())" -
Copy the script to
ch04-first-agent/out/my_agent.py. In your copy:- use your model ID;
- write the calculator's description yourself, before reading the author's;
- choose your own value for
MAX_STEPSand say why in a one-sentence comment; - make the loop raise when the cap is reached;
- return an answer only when the stop reason is
end_turn, and treat any other stop reason that is nottool_useas a recorded failure; - build the reply by joining the text blocks, and collect tool requests by filtering on block type.
-
Save the trace as
ch04-first-agent/out/trace.jsonl, one line per model call:{"ts": "...", "run": "six", "prompt": 0, "step": 1, "model": "...", "stop_reason": "tool_use", "tool_calls": [{"name": "lookup_stock", "input": {"query": "headset"}, "result": "..."}], "usage_input_tokens": 0, "usage_output_tokens": 0}and one closing line per question:
{"ts": "...", "run": "six", "prompt": 0, "record_type": "outcome", "steps": 2, "exit": "end_turn", "max_steps": 5, "answer": "..."}promptcounts from zero.exitis"end_turn","cap"when your loop raised, or the stop reason that ended it otherwise. -
Run the six questions.
- Now make the cap fire. Run the webcam-value question once more with the step cap set to one, and write its lines with
"run": "forced". Its outcome line must say"exit": "cap", and your terminal must show the exception. Why that question and that cap: its answer needs the stock record, which only the lookup tool holds, so the first reply has to be a tool request, and a cap of one stops the loop before it can go round again. A cap of "one less than it took last time" is not safe, because a repeat can find a shorter path. - Take the keyboards-or-headsets question and change a copy of your loop to read only the first content block. Run it once and write down, in
ch04-first-agent/out/notes.md, what came back. This is Lesson 4's first failure on your own machine. - Compare your trace with the chapter's table, question by question, and add three sentences to
notes.md: where yours differs, which exit each run took, and what your cap's headroom was.
Check
This step needs no key and no network:
python ch04-first-agent/ch4_course_check.py
It reads ch04-first-agent/out/trace.jsonl and ch04-first-agent/out/prediction.json in the formats above.
Pass when
ch4_course_check.py prints Module 4 check: PASS and exits with code 0, and you can tick the checklist below. The script passes on any step counts. It checks that:
- every model call of the six questions and the forced run has a line with its stop reason, its tool requests with their full inputs and results, and its token usage;
- each question has one outcome line whose step count and exit agree with its call lines;
- all six questions ended on
end_turn: none was cut off atmax_tokensand none hit your cap; - the forced run used a cap of one and ended at the cap, with exactly one call;
- your predictions are six whole numbers dated before your first call.
The script reads your trace, not your code. Three things only you can check:
Then mark the module done.
If the check fails
The script lists every problem with the number of the check that caught it. Do not edit trace.jsonl; change your loop and run it again. The course's own first two attempts failed on the second and third rows below.
| Message begins | What it means | What to do |
|---|---|---|
Can't check yet |
A file is missing or is not valid JSON (exit code 2) | Write prediction.json first, then run your agent |
Question N stopped on 'max_tokens' |
The reply was cut off, possibly by thinking | Raise max_tokens and rerun; a long, tool-free answer needed more than a thousand tokens on the course run |
The forced run |
The forced run did not end at a cap of one | Set the cap to one on the webcam-value question; a cap of "one less than last time" can be beaten by a shorter path |
stopped on 'tool_use' with no tool calls logged |
Your loop logged only some requests, or none | Log every tool request block with its input |
doesn't recompute |
An outcome line disagrees with the call lines | Rerun; do not edit the file |
hit your step cap |
One of the six questions needed more steps than your cap | Raise the cap and say why in its comment |
Problems the script cannot see but your terminal will show:
| What you see | What it means | What to do |
|---|---|---|
| An error naming tool request ids without a result | A response held more than one request and your loop answered only some | Build one result per request block |
| An attribute error on the first content block | The first block is not text | Filter blocks by type |
| The policy question calls a tool | The calculator's description invites it | Edit the description first, then the tool's name |
Two recorded runs to compare yours with. The book's run took 2, 2, 3, 2, 4, 1 steps for the six questions with a cap of 5 that never fired (BOOK.md, Measured findings, claude-opus-4-6); a later harness built from the chapter's code reproduced that table on 18 of 18 prompt runs. The course's passing run took 2, 1, 3, 2, 3, 1 on claude-opus-5 (runs/ch4-course-check-2026-11-20-run3.txt). Yours will differ somewhere. Say where, and why you think so.
Record your result
Run ch4_course_check.py on your machine, then paste everything it printed. The page reads the first line
(Module 4 check: PASS or FAIL) and keeps the result in this browser. Do not paste an API key; the script never prints one.
No result recorded yet.