What would change your mind?
Three short quests on the Think Clearly trail. Each asks for a first judgment, a short reading, an attempt, a comparison with an annotated model, and a revision. Each ends with something saved, and you can stop after any of them.
Self-assessed, with no credential. The page never marks what you write. You compare your answer with a model and a short list of criteria, and you tick what your answer shows. Your first attempt and your revision are both kept, in this browser only, each with the time it was saved (UTC).
Times on this page are design estimates, not measured times: about 20 minutes a quest.
Quest 1: Separate the report from the evidence
Output: a short explanation of why several causes remain possible.
An invented case from Outage casebook
The wiki that answers pings
An office wiki on one small web server.
“The wiki is down for me. I pinged it and it answers, so it isn't the network.”
What is known
- Last month the wiki was given a new, shorter name.
- A ping to the wiki's name gets replies.
- The browser shows an error page instead of the wiki.
1. Commit about 2 minutes
2. Learn about 4 minutes
- Why we explain too quickly, on Observe, Define, Represent: loaded statements beside plain observations, and the two-column practice that follows.
- Drill A: the ping works, but the website does not, on Evidence-Based Troubleshooting: what a successful ping establishes, and what it leaves untested.
- Optional: play a round of Outage casebook. It draws its cases at random, so this one may or may not come up.
3. Attempt about 6 minutes
4. Compare about 3 minutes
An annotated model
Observed. A ping to the wiki's name gets replies. The browser shows an error page. The wiki was renamed last month.
Each is something seen. None of them says why.
Concluded. “So it isn't the network.”
The ping got replies. An echo reply shows that two points can exchange ICMP messages, now. The conclusion went further than the ping: the ping never touched name lookup, the web port or the certificate.
Why several causes remain possible. A wrong address, a web service that is not answering, and a certificate that does not list the new name all fit the report. In each case the ping gets replies and the browser shows an error, so nothing observed so far tells them apart.
It names the rivals, ties each to the observations, and does not pick a winner. The game's debrief for this case says the same: “A wrong address, a dead web service and a certificate name all fitted the report.”
5. Revise about 3 minutes
Improve your attempt against the criteria. Your first attempt stays saved as it was.
Quest 1 saved: your first judgment, your first attempt and your revision. This is a good place to stop. When you come back, the page opens where you left off.
Quest 2: Choose a discriminating test
Output: one chosen test, its purpose, and a revised set of hypotheses.
Another invented case from Outage casebook
Wi-Fi that drops at lunchtime
A home network: one router, a few laptops and phones.
“The Wi-Fi drops almost every day around half past twelve. It must be the internet provider.”
What is known
- Each drop lasts a few minutes, then everything reconnects.
- Every phone and laptop on Wi-Fi drops at the same time.
- The router sits on a shelf in the kitchen.
Four possible causes
- Provider outage: the internet provider's line drops at lunchtime.
- Router restart: the router overheats or crashes and restarts.
- Interference at home: something in the house, such as a kitchen appliance, disrupts the 2.4 GHz radio band at lunchtime.
- Neighbour's network: a neighbour's busy network on the same channel crowds yours at lunchtime.
1. Commit about 2 minutes
2. Learn about 4 minutes
- Predict before you observe and Strong and weak tests, on Hypotheses, Predictions and Tests: write what each explanation expects before you look, and prefer a test whose result differs sharply with the explanation.
- Drill D: an event preceded a failure, on Evidence-Based Troubleshooting: timing alone does not settle a cause.
3. Attempt about 6 minutes
4. Compare about 3 minutes
The result of your test
The cabled laptop stays online throughout. The Wi-Fi devices drop.
The router has been up for 19 days.
The provider reports no known faults in your area.
The stream stalls within seconds and recovers when the oven stops.
The 5 GHz laptop stays connected. Devices on 2.4 GHz drop.
No drops on any of the three days. They return on the fourth, when the oven is used again.
An annotated model: what each test rules out
| Test | Rules out | Why |
|---|---|---|
| Cable at lunchtime | Provider, router restart | A provider outage or a router restart would cut the cable too. Home interference and a neighbour both still fit. |
| Uptime counter | Router restart | A restart resets the counter. This rules out a restart, and only that. |
| Ring the provider | Nothing | No reported faults is weak evidence. Every hypothesis survives. |
| Microwave at 4 pm | Nothing | It shows the oven can disrupt the Wi-Fi, not that the lunchtime drops come from it. |
| 5 GHz at lunchtime | Provider, router restart | Only the 2.4 GHz band suffers. Home interference and a neighbour on your channel both fit that. |
| No microwave for three lunchtimes | Provider, router restart, neighbour | Remove the suspect and the effect disappears; restore it and the effect returns. A provider, a router fault or a neighbour would not follow your oven. |
The strongest single test is the one whose possible results differ most between the causes. Here, leaving the oven unused splits all four in one move; the cabled laptop splits two from two. A test that every cause predicts the same way (ring the provider, or the oven at 4 pm) tells you almost nothing about the lunchtime drops, however reassuring it feels.
In this bounded puzzle, the fewest tests is a fair score. In real work, cost, risk and how easily a test can be undone count too: Hypotheses, Predictions and Tests asks which observation would reduce your uncertainty the most “for the least cost and risk”. Three lunchtimes without the microwave is cheap and easy to undo; a test that cuts service for everyone would not be.
5. Revise about 3 minutes
Choose again if the model changed your mind, then write the hypotheses that survive. Your first attempt stays saved as it was.
Quest 2 saved: a chosen test, its purpose, what each result would mean, and the hypotheses that survive. A good place to stop.
Quest 3: Apply the method elsewhere
Output: the observations, the causal claim, two alternative explanations and one improved comparison.
A fictional case, invented for this exercise
The reading room
A town opened a reading room. The following month, twenty visitors scored higher on a vocabulary test than twenty non-visitors. The mayor says the room caused the improvement.
No real town, room or test is described.
1. Commit about 2 minutes
2. Learn about 4 minutes
- A cause is a comparison with what did not happen, on Cause and Effect: a causal effect compares the outcome with the outcome that would have followed without the treatment.
- Confounding: groups that were never comparable and Why randomisation works, on the same page: why the reason someone chose a treatment can mislead, and what assignment by chance buys.
3. Attempt about 6 minutes
4. Compare about 3 minutes
An annotated model
Observations. A reading room opened. A month later, twenty visitors scored higher on a vocabulary test than twenty non-visitors.
That is all the case reports. It gives no scores from before the room opened, no size for the gap, and nothing about how the forty people were chosen.
The claim. The room caused the visitors' higher scores: had it never opened, they would have scored lower.
“Improvement” suggests scores went up, but nobody was measured twice; the case compares two groups at one time. Stating the claim as a comparison with what did not happen shows what the evidence would need to reach.
Alternatives (any two of these, or others that fit):
- People who chose to visit already had larger vocabularies, or read more, before the room opened.
- The gap is chance: twenty people a side is a small sample, and the case does not say how large the difference was.
- The two groups sat the test differently, at another time or place, so the observation itself may mislead.
Each would produce the same reported gap even if the room had no effect. The first is the pattern Cause and Effect calls confounding: the reason for the “treatment” may also raise the outcome, so the groups were never comparable.
An improved comparison. Measure the vocabulary of the same people before the room opens and again some months later, visitors and non-visitors alike, and compare how much each group changed. Better still, if places in a reading programme are limited, give them out by lot and compare those who got a place with those who did not.
Measuring before and after takes account of where each person started; it still cannot rule out that visitors would have improved faster anyway. Assignment by lot is what makes the groups comparable in the way Why randomisation works describes.
5. Revise about 3 minutes
Improve your attempt against the criteria. Your first attempt stays saved as it was.
Quest 3 saved. All three quests are done. See what happened.
What happened
You separated a report from its evidence, chose a test and said what each result would mean before seeing it, and carried the same method to a case with no network in it. Your first attempts and your revisions are saved in this browser.
Your three outputs, as revised
- Quest 1: why several causes remain possible
- Quest 2: the test you chose
- Quest 2: its purpose
- Quest 2: the hypotheses that survive
- Quest 3: two alternatives
- Quest 3: the improved comparison
One question to take away: when two explanations predict the same result, what should you do next?
What could come next
- Turn the method on a belief of your own with the Academy's cross-examination, the trail's next step.
- Put a number on how sure you are with Judgment Under Uncertainty, the trail's last step.
- Practise choosing tests on more invented cases in Outage casebook.
Or finish for now. Nothing else is needed, and everything above stays saved.
A case you have not seen
This one is not practice: no model is shown, and it is kept apart from the three quests. Try it now, or in a few days; waiting a few days is a choice for this pilot, not a proven best schedule.
A fictional case, invented for this exercise
A school team began drinking a new sports drink halfway through the season. It won seven of its last ten games, after losing six of its first ten. The coach says the drink made the difference.
Use the criteria from Quest 3: observations with no causal words, the claim as a comparison with what would have happened otherwise, two alternatives that fit the same record, a comparison that says who, on what, when, and how the groups are made alike, and no verdict on whether the drink worked.
Saved. Check it against the criteria yourself, then set it beside your Quest 3 revision. One attempt on one case shows what you did once; it is not a certificate.