Measured Academy Measured Academy
Français Pilot Progress 0

Module 4 · Hands-on test

Two tools, a cap that raises, and a trace you can read

You run this on your own machine with your own API key. Nothing you run is sent to this site.

Get the files

Everything this test asks you to run is in one download, laid out exactly as in the companion repository, so every command in the steps works from the unzipped folder.

Download the Module 4 kit (zip, 7 files)

  • ch04-first-agent/ch03_trace.py
  • ch04-first-agent/ch4_course_agent.py
  • ch04-first-agent/ch4_course_check.py
  • ch04-first-agent/course-agent-2026-11-20-attempt3.txt
  • ch04-first-agent/course-agent-2026-11-20-attempt3.jsonl
  • ch04-first-agent/course-prediction-2026-11-20-attempt3.json
  • ch04-first-agent/course-check-2026-11-20-attempt3.txt

Unzip it, open a terminal in module-4-kit, create a Python 3.12 virtual environment, and install what Modules 1 to 4 need:

pip install anthropic==1.2.0 pydantic==2.13.5

Those two pins are read from the kit's requirements.txt, which also lists packages only later modules use. The companion repository is not open to pilot learners yet, so use this download wherever the steps say "companion repo".

Try the check before you spend anything. This runs it on the course's own recorded run and needs no key:

python ch04-first-agent/ch4_course_check.py --trace ch04-first-agent/course-agent-2026-11-20-attempt3.jsonl --prediction ch04-first-agent/course-prediction-2026-11-20-attempt3.json

It should print Module 4 check: PASS. That is the course's result, not yours: do not record it below.

What you will do

Add a calculator as a second tool to the chapter's stock-lookup agent, predict how many steps each of the six questions will take, run them, and read your trace against the chapter's table. Then make the step cap fire on purpose and confirm it raises.

This course is for learners aged 18+.

Cost

You use your own API key, so you pay for this run.

What Course run, claude-opus-5 (passing attempt) Book run, claude-opus-4-6
Model calls 13: twelve for the six questions, one forced not recorded
Input tokens 7,730 not recorded: the run logged no usage
Output tokens 2,430, of which 666 thinking not recorded
Cost of that run $0.10 no figure
Source BOOK.md, Measured findings; runs/ch4-course-check-2026-11-20-run3.txt COST.md lists Ch 4 among the runs that logged no usage

The course needed three attempts to pass (Module 4, Lesson 2). The first two cost $0.10 and $0.13. If your run fails its check, a rerun costs about as much again. No single question can make more calls than your cap, and each call sends both tool definitions and the whole message list so far, so later steps cost more input than earlier ones.

Steps

Work from the unzipped kit (the same files as the companion repo's ch04-first-agent/), from its top folder. Your files go in ch04-first-agent/out/. The kit also holds the course's own harness, ch4_course_agent.py, and its passing run. Build yours first; open those afterwards to compare.

  1. Read the chapter's exercise, then ch04-first-agent/ch03_trace.py. The script keeps the number it was written under. It is the author's harness and it has two things you will change: when it runs out of steps it falls through instead of raising, and it prints its trace instead of saving it.
  2. Write your predictions and save them as ch04-first-agent/out/prediction.json before any call:

    {
      "predicted_steps": [0, 0, 0, 0, 0, 0],
      "predicted_at": "2026-01-01T00:00:00+00:00"
    }
    

    One whole number per question, in the order of the script's PROMPTS list. Set predicted_at to the current UTC time. This prints it: python -c "from datetime import datetime, timezone; print(datetime.now(timezone.utc).isoformat())"

  3. Copy the script to ch04-first-agent/out/my_agent.py. In your copy:

    • use your model ID;
    • write the calculator's description yourself, before reading the author's;
    • choose your own value for MAX_STEPS and say why in a one-sentence comment;
    • make the loop raise when the cap is reached;
    • return an answer only when the stop reason is end_turn, and treat any other stop reason that is not tool_use as a recorded failure;
    • build the reply by joining the text blocks, and collect tool requests by filtering on block type.
  4. Save the trace as ch04-first-agent/out/trace.jsonl, one line per model call:

    {"ts": "...", "run": "six", "prompt": 0, "step": 1, "model": "...",
     "stop_reason": "tool_use",
     "tool_calls": [{"name": "lookup_stock", "input": {"query": "headset"}, "result": "..."}],
     "usage_input_tokens": 0, "usage_output_tokens": 0}
    

    and one closing line per question:

    {"ts": "...", "run": "six", "prompt": 0, "record_type": "outcome",
     "steps": 2, "exit": "end_turn", "max_steps": 5, "answer": "..."}
    

    prompt counts from zero. exit is "end_turn", "cap" when your loop raised, or the stop reason that ended it otherwise.

  5. Run the six questions.

  6. Now make the cap fire. Run the webcam-value question once more with the step cap set to one, and write its lines with "run": "forced". Its outcome line must say "exit": "cap", and your terminal must show the exception. Why that question and that cap: its answer needs the stock record, which only the lookup tool holds, so the first reply has to be a tool request, and a cap of one stops the loop before it can go round again. A cap of "one less than it took last time" is not safe, because a repeat can find a shorter path.
  7. Take the keyboards-or-headsets question and change a copy of your loop to read only the first content block. Run it once and write down, in ch04-first-agent/out/notes.md, what came back. This is Lesson 4's first failure on your own machine.
  8. Compare your trace with the chapter's table, question by question, and add three sentences to notes.md: where yours differs, which exit each run took, and what your cap's headroom was.

Check

This step needs no key and no network:

python ch04-first-agent/ch4_course_check.py

It reads ch04-first-agent/out/trace.jsonl and ch04-first-agent/out/prediction.json in the formats above.

Pass when

ch4_course_check.py prints Module 4 check: PASS and exits with code 0, and you can tick the checklist below. The script passes on any step counts. It checks that:

  • every model call of the six questions and the forced run has a line with its stop reason, its tool requests with their full inputs and results, and its token usage;
  • each question has one outcome line whose step count and exit agree with its call lines;
  • all six questions ended on end_turn: none was cut off at max_tokens and none hit your cap;
  • the forced run used a cap of one and ended at the cap, with exactly one call;
  • your predictions are six whole numbers dated before your first call.

The script reads your trace, not your code. Three things only you can check:

Then mark the module done.

If the check fails

The script lists every problem with the number of the check that caught it. Do not edit trace.jsonl; change your loop and run it again. The course's own first two attempts failed on the second and third rows below.

Message begins What it means What to do
Can't check yet A file is missing or is not valid JSON (exit code 2) Write prediction.json first, then run your agent
Question N stopped on 'max_tokens' The reply was cut off, possibly by thinking Raise max_tokens and rerun; a long, tool-free answer needed more than a thousand tokens on the course run
The forced run The forced run did not end at a cap of one Set the cap to one on the webcam-value question; a cap of "one less than last time" can be beaten by a shorter path
stopped on 'tool_use' with no tool calls logged Your loop logged only some requests, or none Log every tool request block with its input
doesn't recompute An outcome line disagrees with the call lines Rerun; do not edit the file
hit your step cap One of the six questions needed more steps than your cap Raise the cap and say why in its comment

Problems the script cannot see but your terminal will show:

What you see What it means What to do
An error naming tool request ids without a result A response held more than one request and your loop answered only some Build one result per request block
An attribute error on the first content block The first block is not text Filter blocks by type
The policy question calls a tool The calculator's description invites it Edit the description first, then the tool's name

Two recorded runs to compare yours with. The book's run took 2, 2, 3, 2, 4, 1 steps for the six questions with a cap of 5 that never fired (BOOK.md, Measured findings, claude-opus-4-6); a later harness built from the chapter's code reproduced that table on 18 of 18 prompt runs. The course's passing run took 2, 1, 3, 2, 3, 1 on claude-opus-5 (runs/ch4-course-check-2026-11-20-run3.txt). Yours will differ somewhere. Say where, and why you think so.

Record your result

Run ch4_course_check.py on your machine, then paste everything it printed. The page reads the first line (Module 4 check: PASS or FAIL) and keeps the result in this browser. Do not paste an API key; the script never prints one.

No result recorded yet.