Technology, AI and Human Judgment

Computers did not arrive as a finished invention. They grew out of questions in logic, communication and control, and the people who built them left behind both theory and warnings. This page follows that thread, explains at a conceptual level how modern AI works, and sets out rules for using it in study so that it strengthens your mind instead of replacing it.

Foundations: logic, computation, information

Boole. In The Laws of Thought (1854), Boole showed that reasoning with classes can be written as algebra over two values. That is the seed of digital circuits. Demonstrated (it is mathematics).

Turing. His 1936 paper defined a machine that follows simple rules on a tape and argued that it can carry out any procedure we would call mechanical. He also proved that the Entscheidungsproblem, a question posed in logic, has no general algorithmic solution; the closely related halting problem (deciding in general whether a program will stop) is its modern restatement. Demonstrated for the halting result; the claim that his machine captures all "effective" computation (the Church-Turing thesis) is Established but is a thesis, not a theorem.

Shannon. His 1948 paper measured information as the uncertainty, or surprise, that a message removes. A fair coin flip carries 1 bit; a coin that lands heads 99 percent of the time carries far less, because the outcome is rarely surprising. The entropy formula, H = -sum of p times log2 p, sets limits on compression and on reliable communication over a noisy channel. Demonstrated Note that Shannon's "information" measures surprise, not meaning; confusing the two is a classic error.

Wiener (The Human Use of Human Beings, 1950) connected feedback, communication and automation, and worried about what happens when machines take over tasks that used to require human judgment. The related formal ideas appear in A Map of Mathematics: Twenty-Five Areas and What Depends on What and Discrete Mathematics and Graphs: Counting, Relations and Networks.

How modern AI works, conceptually

A neural network is a large function with adjustable numbers (weights) tuned to reduce error on examples; backpropagation, popularized by Rumelhart, Hinton and Williams (1986), is the method for computing how to adjust them. Large language models use the transformer architecture (Vaswani et al., 2017), in which attention lets each word in a text weigh its relation to the others. Kaplan et al. (2020) reported that performance improved smoothly as model size, data and computing power grew. Models are then often tuned to follow instructions using human feedback (Ouyang et al., 2022), and are measured against evaluations. Established as a description of how these systems are built. What is happening inside them, and whether it deserves words like "understanding," is Aporetic, and honest researchers disagree.

Two cautions about fact versus analogy. First, "the model learns like a brain" is an analogy; artificial neurons are loose abstractions. Second, a fluent answer is a fact about the model's training, not evidence that the answer is true.

Limits and critics of machine reason

• Goedel (1931) proved that any consistent, effectively axiomatized system rich enough for arithmetic contains true statements it cannot prove. Demonstrated Arguments that this shows machines cannot think are Aporetic; the theorem itself says nothing directly about minds. Hofstadter's Goedel, Escher, Bach explores self-reference as an analogy for consciousness, and should be read as a speculative essay.
• Dreyfus argued that skilled human action depends on embodied, situational know-how that rule-following programs cannot capture. His predictions about particular programs were partly overturned by later machine learning, so his position is Provisional.
• Weizenbaum, after seeing people confide in his simple ELIZA program, argued in Computer Power and Human Reason that some tasks should not be handed to machines even if they could do them, because they require human responsibility and care.
• Polanyi observed that much knowledge is tacit: you can ride a bicycle without being able to state the rules. Learning from text alone leaves that residue out.

Alignment: the metric is not the good

Systems optimize what they are told to measure. Amodei and colleagues (2016) list failure modes including reward hacking, where a system exploits a loophole in its objective, and negative side effects. The deep point is older than AI: any proxy, whether a test score, a click or a quarterly target, can be maximized while the thing it stood for is lost. This links to the incentive problems discussed in Systems, Decisions and Robust Design.

Critics of technique

Ellul (The Technological Society) argued that modern societies pursue efficiency in every domain until technique becomes self-justifying. Heidegger's essay "The Question Concerning Technology" (1954) claimed technology frames the world as raw material to be ordered and used. Postman (Amusing Ourselves to Death, Technopoly) argued that each medium changes what counts as serious thought. Huxley's Brave New World feared distraction and pleasure; Orwell's Nineteen Eighty-Four feared surveillance and force. Lewis's The Abolition of Man asked whether conquering nature ends in some people controlling others without any shared moral standard. These are interpretive arguments, Provisional at best; read them as questions to test, not conclusions to adopt.

Using AI as tutor, adversary and tool

The rule is simple: use AI to make your thinking harder, not to excuse it. Good uses:

• Ask for Socratic questions about a text you have already read.
• Ask for the strongest objection to your reconstruction of an argument.
• Ask it to find hidden premises, then check them yourself.
• Ask for an oral examination, answering aloud before looking anything up.
• Generate practice problems and compare methods.

Never use it before your first reading of the text, and avoid these bad uses:

• Replacing the first reading with a summary.
• Producing notes you have not earned by thinking.
• Treating a summary as understanding.
• Deciding anything important without independent verification.

The reason is practical. Experiments on retrieval practice (Roediger and Karpicke, 2006) found that recalling material from memory produces better long-term retention than rereading it, which is Established for the memory tasks studied; extending that to broader skills is a reasonable inference, Provisional. If the tool does the effortful part, you keep the feeling of progress and lose the progress. The routine in The Weekly Loop: Read, Reconstruct, Make, Defend, Log builds these rules into a weekly rhythm. For coding specifically, see Working with AI Coding Agents: A Disciplined Loop and, for tool-using systems, From Assistant to Agent: Tools, MCP and Evaluation.

Testing a model yourself

Treat any model as a witness to cross-examine. Try these checks:

1. Hallucination. Ask about obscure but checkable facts and verify each against a primary source (Ji et al., 2023, survey the causes).
2. Calibration. Ask for confidence with each answer and see whether the 80 percent claims are right about 80 percent of the time (Guo et al., 2017, examine this for neural networks).
3. Bias. Swap names, genders or places in an otherwise identical prompt and compare responses.
4. Prompt sensitivity. Rephrase the same question five ways and note whether the answers change.

Keep a log of results; a handful of your own tests teaches more than any benchmark headline.

Attention, freedom and what must stay human

Tools shape attention, and attention shapes character. A learner can protect both by setting limits on notifications and feeds, reading long texts without a screen that can interrupt, and doing some work by hand on purpose. Larger projects, such as building a small program, threat-modelling a system or measuring how often it fails, give you evidence about technology that opinion cannot. What must remain human is the choice of ends and the responsibility for outcomes, the subject of Reasoning for What Ends? Wisdom, Character and the Examined Life.

Try this

1. Entropy by hand. Compute the entropy in bits of a coin with 90 percent heads. Compare it with a fair coin and explain in a sentence why it is lower.
2. Cross-examine a model. Run the four tests above with ten questions each. Write one paragraph on where the model was trustworthy and where it was not.
3. Earn the note. Read a short text, write your own summary from memory, and only then ask an AI to critique it. Record what it caught that you missed.

Further reading

• Shannon, "A Mathematical Theory of Communication" (1948).
• Turing, "Computing Machinery and Intelligence" (1950).
• Russell and Norvig, Artificial Intelligence: A Modern Approach (4th ed., 2020).
• Weizenbaum, Computer Power and Human Reason (1976).
• Postman, Technopoly (1992).
• Lewis, The Abolition of Man (1943).
• Huyen, AI Engineering (O'Reilly, 2025).