Observe, Define, Represent: The First Three Moves

Before you can reason well about a problem, three things have to go right. You have to say what you actually saw, you have to say what your words mean, and you have to put the problem in a form where its structure shows. These are the first three moves of careful thinking, and most failed arguments and failed investigations went wrong at one of them, not at the clever part that followed.

Why we explain too quickly

People convert raw events into interpretations so fast that they do not notice the conversion. "The server is overloaded", "the application is broken", "my colleague is careless" and "that argument is wrong" all sound like descriptions. Each is already a conclusion.

Compare the loaded statement with an observation:

The right-hand column is longer and duller, and far more useful. It rules out fewer possibilities, so it leaves room for the real cause. Established as a principle of scientific method; the habit of explaining prematurely is familiar to anyone who has debugged or investigated anything, and separating observation from interpretation is a standard discipline in science and engineering.

Submitting to reality

Claude Bernard, in his Introduction to the Study of Experimental Medicine (1865), insisted that the investigator must submit to what the experiment shows, even when it contradicts the theory he started with. A hypothesis is a question put to nature, not a verdict to be defended. Francis Bacon, in the Novum Organum (1620), had earlier catalogued the habits that distort observation, which he called idols:

• Idols of the Tribe: weaknesses built into human nature, such as seeing more order than exists.
• Idols of the Cave: distortions from one person's temperament, training and history.
• Idols of the Marketplace: errors that arise from words, and the way language shapes thought.
• Idols of the Theatre: errors absorbed from systems of philosophy and fashionable doctrines.

Treat these as a checklist for your own mind. The marketplace idols lead directly to the next move.

Practice: two columns

Take any note, report or argument. Draw two columns, one headed "Observed" and one "Interpreted". Move each phrase into the right column. Words like "obviously", "clearly", "careless", "broken" and "unstable" almost always belong on the right. Then rewrite the right-hand items as claims you could check.

Definition: what exactly do we mean?

Socrates asked "what is it?" of courage, piety and virtue, and found that people could use a word fluently without being able to say what it meant (see The Seven Liberal Arts, Hands-On: Logic, Proof and the Quadrivium for the dialogues). Aristotle gave a classical form of definition: name the kind (the genus) and then what sets the thing apart within that kind (the differentia), as in his discussions in the Topics and Posterior Analytics.

For everyday and technical work, an operational definition is often more useful. The physicist P. W. Bridgman argued in The Logic of Modern Physics (1927) that a concept should be defined by the operations used to measure it. Apply this to vague words:

• Unstable: what measurement, crossing what threshold, counts as instability?
• Slow: slower than what, measured where, for which users?
• Secure: secure against which threat, under what assumptions, with what residual risk?
• Reasonable: which premise, inference or standard of evidence is meant?
• Healthy (of a food): by which criterion and which study? See Reading Nutrition Evidence: What a Study Can and Cannot Show for why this matters.

Verbal disagreements

Many disputes are verbal. William James's example, in Pragmatism (1907), is a man who walks around a tree while a squirrel keeps to the opposite side of the trunk. Has the man gone around the squirrel? It depends on what "going around" means (passing to the north, east, south and west of it, which he did, or passing in front of it, then to its right, behind it and to its left, which he did not, since it kept turning with him). Once the meaning is fixed, nothing is left to dispute. When you find yourself in a stubborn disagreement, ask each side to define the key term and see whether the argument survives. Sometimes it does, and then it is a real disagreement worth having. Established: this is a standard move in analytic philosophy.

Representation: choose a form that shows structure

The difficulty of a problem depends partly on how it is presented. Herbert Simon argued that solving a problem often amounts to representing it so that the solution becomes transparent. A few familiar cases:

Two real examples. In 1854, John Snow plotted cholera deaths on a map of Soho and the pumps around them, making the cluster around one pump visible in a way that a list of names was not. Demonstrated as a picture of the data, though Snow's case rested on more than the map. And Gigerenzer and Hoffrage (1995) found that people reason about conditional probabilities much more accurately when the information is given as natural frequencies ("10 out of 1,000") than as percentages. Established in the experimental literature on probability reasoning.

Mathematics as a growing vocabulary

Each branch of mathematics adds a way of representing things. Geometry shows relations in space and invariants. Algebra shows structure through symbols. Linear algebra gives state spaces. Probability handles uncertainty, calculus handles change, graph theory handles networks, information theory handles communication, and control theory handles feedback. You do not need to master all of them. You need enough of each that, when a problem resists one language, you can try another. A Map of Mathematics: Twenty-Five Areas and What Depends on What lays out the branches and what depends on what.

Where the next moves begin

Once you have a clean observation, a defined problem and a good representation, you can start explaining, and not before. Hypotheses, Predictions and Tests covers that step. For the same discipline in practical diagnosis, see Evidence-Based Troubleshooting: Separating Explanations. For writing that keeps the three moves visible to a reader, see Technical Writing as Reasoning. The overview of this whole wing is Liberal Arts and Reasoning: Start Here.

Try this

1. Take a complaint or report you have read recently. Split it into two columns, observation and interpretation, then rewrite each interpretation as something you could test.
2. Choose one vague word you use at work or home ("efficient", "fair", "stable"). Write an operational definition: what would you measure, and what result would count?
3. Redraw one dense paragraph (a news story, a contract clause, a bug report) as a diagram: a timeline, a premises-and-conclusion list, or a causal graph. Note what the diagram revealed that the paragraph hid.

Further reading

• Francis Bacon, Novum Organum, Book I, aphorisms 38 to 68.
• Claude Bernard, An Introduction to the Study of Experimental Medicine, Part One.
• P. W. Bridgman, The Logic of Modern Physics, chapter 1.
• Herbert Simon, The Sciences of the Artificial.
• Gerd Gigerenzer, Calculated Risks (Simon and Schuster, 2002).
• William James, Pragmatism, Lecture II.