Module 2 · Hands-on test
Collapse the chain, then write the decision record
You run this on your own machine with your own API key. Nothing you run is sent to this site.
Get the files
Everything this test asks you to run is in one download, laid out exactly as in the companion repository, so every command in the steps works from the unzipped folder.
Download the Module 2 kit (zip, 9 files)
ch02-stack/ch2_course_collapse.pych02-stack/ch2_course_check.pych02-stack/ch2_collapse.pych02-stack/run-2026-09-07.txtch02-stack/run-2026-09-07.jsonlch02-stack/course-run-2026-11-20.txtch02-stack/course-run-2026-11-20.jsonlch02-stack/course-prediction-2026-11-20.jsonch02-stack/course-check-2026-11-20.txt
Unzip it, open a terminal in module-2-kit, create a Python 3.12 virtual environment, and install what Modules 1 to 4 need:
pip install anthropic==1.2.0 pydantic==2.13.5
Those two pins are read from the kit's requirements.txt, which also lists packages only later modules use. The companion repository is not open to pilot learners yet, so use this download wherever the steps say "companion repo".
Try the check before you spend anything. This runs it on the course's own recorded run and needs no key:
python ch02-stack/ch2_course_check.py --run ch02-stack/course-run-2026-11-20.jsonl --prediction ch02-stack/course-prediction-2026-11-20.json --part a
It should print Module 2 check: PASS (Part A only). That is the course's result, not yours: do not record it below.
What you will do
Two parts, both checked by one script.
Part A, a run. Route the chapter's 10 tickets with the supplied three-call chain and with a single call, against an answer key you write first, using the course harness. Predict both scores before you run.
Part B, a decision record. Choose a stack for the scenario below and write down, for every piece, why you use it now or what would make you add it.
This course is for learners aged 18+.
Cost
You use your own API key, so you pay for this run.
| What | Course run, claude-opus-5 |
Book run, claude-opus-4-6 |
|---|---|---|
| Calls | 40 (30 chain, 10 single) | 40 |
| Input tokens | 8,142 | 2,599 |
| Output tokens | 9,058, of which 3,369 thinking | 799 |
| Cost of that run | $0.27 | $0.03 |
| Source | BOOK.md, Measured findings; runs/ch2-course-collapse-2026-11-20.txt |
COST.md, Ch 2 row; runs/ch2-collapse-2026-09-07.txt |
Expect a figure near the course run's if you use a current model. Yours depends on the model you choose and the prices on the day. Part B costs nothing.
Steps
Work from the unzipped kit (it holds the same files as the companion repo's ch02-stack/), from its top folder. Your files go in ch02-stack/out/.
Part A
- Read the ten tickets and the four queues in the chapter's exercise. Do not open
ch2_collapse.pyorch2_course_collapse.pyyet: both contain the author's answer key and single-call prompt. -
Write your answer key, one queue per ticket in the chapter's order, and predict how many tickets each version will match exactly. The harness saves them with the time, before any call:
python ch02-stack/ch2_course_collapse.py predict --chain <0-10> --single <0-10> --key billing,billing,...Leave out
--keyonly if you want to use the author's key. -
Run it on your own key and a current model ID:
python ch02-stack/ch2_course_collapse.py run --model <model id>It makes the chapter's thirty chain calls and ten single calls, reads the first text block of each reply (Module 1, Lesson 2), logs thinking tokens, and stops if it reaches its call or token limit. It writes
ch02-stack/out/run.jsonl. -
Now open the harness and compare your key with the author's. Disagree with a row, not with a total. Then compare your run with the course run in
ch02-stack/course-run-2026-11-20.txt, ticket by ticket. - Optional, for a second run: copy the harness, replace
SINGLE_PROMPTwith a prompt you write (the four queue names, a line on what each owns, an instruction to reply with one word), and run your copy. That is the exercise as the chapter sets it.
Part B
Scenario: a four-person team runs an internal help desk. They want a tool that reads each incoming ticket, routes it to one of four queues and drafts a first reply for a human to edit. Volume is a few hundred tickets a day. Nothing is sent without a person approving it. There is a wiki of procedures the drafts could cite later, but it is out of date and nobody has asked for citations yet.
Write ch02-stack/out/stack-decision.md containing one Markdown table with exactly these columns:
| Component | Decision | Reason or trigger |
| --- | --- | --- |
| Provider SDK | use | ... |
| Agent framework | defer | ... |
- At least four rows.
Decisionisuseordefer.- A
userow gives the present requirement that needs the piece. - A
deferrow gives a trigger: something observable that would have to happen, twice, before you add it.
Check
This step needs no key and no network:
python ch02-stack/ch2_course_check.py
Pass when
ch2_course_check.py prints Module 2 check: PASS and exits with code 0, and you can tick every line of the checklist. The script passes on any score. It checks that:
- the run file has its header and every call record with all fields;
- both versions ran over all ten tickets: three chain stages and one single call per ticket;
- the tickets and queues are the module's, on one model, and the key in the run is the key in your prediction;
- the scores in the header recompute from the call records, so the file was not edited;
- token usage is logged for every call, no answer is empty and none stopped on
max_tokens; - your predictions are whole numbers in range and dated before your first call;
- your decision record has the table, at least four rows, a valid decision on each, at least one
defer, and a reason or trigger on every row.
Checklist, which you judge yourself until Measured Lab grades it:
Then mark the module done.
If the check fails
The script lists every problem with the number of the check that caught it. Do not edit run.jsonl; rerun the step that produced it.
| Message begins | What it means | What to do |
|---|---|---|
Can't check yet |
A file is missing or is not valid JSON (exit code 2) | Run the predict and run steps, and write stack-decision.md, all in ch02-stack/out/ |
Expected ... calls |
A version did not run over every ticket, or the harness stopped at a limit | Rerun the harness |
key in the run differs |
You changed your key after predicting | Predict again with the key you will run, then run again |
doesn't recompute |
The header and the call records disagree | Rerun; do not edit the file |
the answer is empty or stopped on 'max_tokens' |
The model spent the budget thinking | Rerun with a larger budget, for example run --model <id> --max-tokens 2048 |
Decision record |
The table is missing, short, or has a row with no reason | Fix stack-decision.md and run the check again |
Two recorded runs to compare yours with, both on the author's key. The book's run on claude-opus-4-6 scored the chain at 4 of 10 exact and 9 of 10 on letters only, and the single call at 10 of 10 (runs/ch2-collapse-2026-09-07.txt). The course run on claude-opus-5 scored the chain at 8 of 10 exact and 9 of 10 on letters only, and the single call at 10 of 10 (runs/ch2-course-collapse-2026-11-20.txt). Yours will differ. Say how, and why you think so.
Record your result
Run ch2_course_check.py on your machine, then paste everything it printed. The page reads the first line
(Module 2 check: PASS or FAIL) and keeps the result in this browser. Do not paste an API key; the script never prints one.
No result recorded yet.