Observe, Define, Represent: The First Three Moves

How to separate observation from interpretation, ask what exactly we mean, and choose a representation that exposes structure.

Before you can reason well about a problem, three things have to go right. You need a plain observation, defined terms, and a form that shows the problem's structure. These are the first three moves of careful thinking. Most failed arguments and investigations went wrong at one of them, not at the clever part that followed. Provisional: a rule of thumb, not a measured rate.

Why we explain too quickly

People convert raw events into interpretations so fast that they do not notice the conversion. "The server is overloaded", "the application is broken", "my colleague is careless" and "that argument is wrong" all sound like descriptions. Each is already a conclusion.

Compare the loaded statement with an observation:

"The network is slow.". Observation: Round-trip time between two points rose from about 12 ms to about 90 ms, starting at a certain time.
"The service keeps failing.". Observation: One kind of request timed out three times in a row; a different kind succeeded each time.
"That argument is wrong.". Observation: The conclusion does not follow from premise 2, because premise 2 says "some", and the conclusion needs "all".

The right-hand column is longer and duller, and far more useful. It rules out fewer possibilities, so it leaves room for the real cause. Established as a principle of scientific method; the habit of explaining prematurely is familiar to anyone who has debugged or investigated anything, and separating observation from interpretation is a standard discipline in science and engineering.

Submitting to reality

Claude Bernard, in his Introduction to the Study of Experimental Medicine (1865), insisted that the investigator must submit to what the experiment shows, even when it contradicts the theory he started with. A hypothesis is a question put to nature, not a verdict to be defended. Francis Bacon, in the Novum Organum (1620), had earlier catalogued the habits that distort observation, which he called idols:

Idols of the Tribe: weaknesses built into human nature, such as seeing more order than exists.
Idols of the Cave: distortions from one person's temperament, training and history.
Idols of the Marketplace: errors that arise from words, and the way language shapes thought.
Idols of the Theatre: errors absorbed from systems of philosophy and fashionable doctrines.

Treat these as a checklist for your own mind. The marketplace idols lead directly to the next move.

Practice: two columns

Take any note, report or argument. Draw two columns, one headed "Observed" and one "Interpreted". Move each phrase into the right column. Words like "obviously", "clearly", "careless", "broken" and "unstable" almost always belong on the right. Then rewrite the right-hand items as claims you could check.

Definition: what exactly do we mean?

Socrates asked "what is it?" of courage, piety and virtue, and found that people could use a word fluently without being able to say what it meant (see Seven Liberal Arts, Hands-On for the dialogues). Aristotle gave a classical form of definition: name the kind (the genus) and then what sets the thing apart within that kind (the differentia), as in his discussions in the Topics and Posterior Analytics.

For everyday and technical work, an operational definition is often more useful. The physicist P. W. Bridgman argued in The Logic of Modern Physics (1927) that a concept should be defined by the operations used to measure it. Apply this to vague words:

Unstable: what measurement, crossing what threshold, counts as instability?
Slow: slower than what, measured where, for which users?
Secure: secure against which threat, under what assumptions, with what residual risk?
Reasonable: which premise, inference or standard of evidence is meant?
Healthy (of a food): by which criterion and which study? In Reading Nutrition Evidence, one trial found flaxseed lowered blood pressure more than placebo: healthy by that measure. A blood-pressure change is still not a measured reduction in strokes.

Verbal disagreements

Many disputes are verbal. William James's example, in Pragmatism (1907), is a man who walks around a tree while a squirrel keeps to the opposite side of the trunk. Has the man gone around the squirrel? It depends on what "going around" means (passing to the north, east, south and west of it, which he did, or passing in front of it, then to its right, behind it and to its left, which he did not, since it kept turning with him). Once the meaning is fixed, nothing is left to dispute. When you find yourself in a stubborn disagreement, ask each side to define the key term and see whether the argument survives. Sometimes it does, and then it is a real disagreement worth having. Established: this is a standard move in analytic philosophy.

Representation: choose a form that shows structure

The difficulty of a problem depends partly on how it is presented. Herbert Simon argued that solving a problem often amounts to representing it so that the solution becomes transparent. A few familiar cases:

Events in order. Useful representation: Timeline.
Delays and backlogs. Useful representation: Queue.
Who talks to whom. Useful representation: Network topology.
Who trusts whom. Useful representation: Trust-boundary diagram.
A philosophical or legal argument. Useful representation: Premises and conclusion.
A probability question. Useful representation: Frequency table (natural counts).
Cause and effect. Useful representation: Directed causal graph.

Two real examples. In 1854, John Snow plotted cholera deaths on a map of Soho and the pumps around them, making the cluster around one pump visible in a way that a list of names was not. Demonstrated as a picture of the data, though Snow's case rested on more than the map. And Gigerenzer and Hoffrage (1995) found that people reason about conditional probabilities much more accurately when the information is given as natural frequencies ("10 out of 1,000") than as percentages. Established in the experimental literature on probability reasoning.

Mathematics as a growing vocabulary

Each branch of mathematics adds a way of representing things. Geometry shows relations in space and invariants. Algebra shows structure through symbols. Linear algebra gives state spaces. Probability handles uncertainty, calculus handles change, graph theory handles networks, information theory handles communication, and control theory handles feedback. You do not need to master all of them. You need enough of each that, when a problem resists one language, you can try another. A Map of Mathematics: Twenty-Five Areas and What Depends on What lays out the branches and what depends on what.

Where the next moves begin

Once you have a clean observation, a defined problem and a good representation, you can start explaining, and not before. Hypotheses, Predictions and Tests covers that step. For the same discipline in practical diagnosis, see Evidence-Based Troubleshooting. For writing that keeps the three moves visible to a reader, see Technical Writing as Reasoning. The overview of this whole wing is Liberal Arts and Reasoning.

Try this

Take a complaint or report you have read recently. Split it into two columns, observation and interpretation, then rewrite each interpretation as something you could test.
Choose one vague word you use at work or home ("efficient", "fair", "stable"). Write an operational definition: what would you measure, and what result would count?
Redraw one dense paragraph (a news story, a contract clause, a bug report) as a diagram: a timeline, a premises-and-conclusion list, or a causal graph. Note what the diagram revealed that the paragraph hid.