Systems Thinking: Feedback, Queues, Information and Dynamics

A system is a set of parts whose arrangement produces behaviour that no single part explains. This page tours the mathematics that lets you say something precise about that behaviour: feedback, queues, information, nonlinear dynamics, signals, causation and decision. It ends with a warning that these are models, and a good model is not the same thing as a discovery about the nature of reality.

If you want the reading-list companion to this page, see Systems, Decisions and Robust Design. For the calculus underneath much of it, see Calculus and Beyond: Change, Structure and Uncertainty.

From structure to behaviour: stocks, flows and loops

Donella Meadows' Thinking in Systems (ch. 1) builds everything from three ideas. A stock is an accumulation you could measure at an instant: water in a bathtub, requests waiting to be served, money in an account. A flow changes a stock over time: the tap and the drain. A feedback loop exists when a stock's level influences the flows that change it.

• A balancing loop pushes a stock toward a goal (a thermostat, a queue that discourages new arrivals as it grows).
• A reinforcing loop amplifies change (compound interest, a panic that causes more panic).

Most surprises come from delays. A balancing loop with a long delay overshoots and oscillates, which is why a shower with a slow tap feels so hard to set (an everyday analogy for the mechanism, not a proof of it). Established The structural claim that behaviour follows from loops, stocks and delays is standard systems dynamics. Meadows' related essay "Leverage Points" ranks intervention points, but that ranking is her judgement, Provisional, not a theorem.

Queues: why averages lie

Queueing theory studies waiting lines. Two results are worth remembering.

Little's Law (Little, 1961): in a stable system, average number in the system equals average arrival rate times average time spent in the system, L = λW. It needs almost no assumptions, which is why it is so useful for sanity checks. Demonstrated (it is a theorem).

Utilization is not linear. For the simplest single-server model (M/M/1: random arrivals, random service times, one server), average time in the system is the service time divided by (1 minus utilization):

A server that is "only" 90% busy means every job spends about ten times its own service time in the system. For this model the 99th-percentile response time is about 4.6 times the average (ln 100), so tail latency is worse still. Dean and Barroso's "The Tail at Scale" shows why this matters in large services: when one user request fans out to many machines, the rare slow response on any one of them becomes the common experience. The M/M/1 numbers are Demonstrated for the model; real systems with bursty or heavy-tailed traffic are usually worse. Harchol-Balter's book is the best practical introduction.

Control theory: regulation and stability

Control theory asks how a system can be steered toward a target despite disturbances. A PID controller computes a correction from three terms, in plain English:

• Proportional: push in proportion to how far you are from the target now.
• Integral: push harder the longer you have been off target, which removes a persistent small offset.
• Derivative: ease off when you are closing fast, which anticipates overshoot.

Picture a heater controlled by a thermostat with a sluggish sensor. Too little proportional action and the room warms sluggishly, often settling short of the target; too much, combined with delay, and the temperature swings around the target with growing amplitude. The integral term can wind up while the heater is saturated and cause a late overshoot. Stability is a trade between speed and swing. Åström and Murray's Feedback Systems develops this properly. Established

Information theory: limits on measurement and communication

Shannon (1948) defined the entropy of a source as H = −Σ p log₂ p bits per symbol. A fair coin carries 1 bit per toss; a coin that lands heads 90% of the time carries about 0.47 bits, because it is more predictable. Entropy sets the limit of lossless compression.

The noisy-channel coding theorem says a channel with random errors has a capacity, and reliable communication is possible at any rate below it by adding structured redundancy (error-correcting codes). A channel that flips each bit with probability 0.11 has capacity of about 0.5 bits per use. The lesson for measurement: noise puts a floor under what a sensor can tell you, and redundancy is how you pay for confidence. Demonstrated (theorems). Calling a society or a brain "a communication channel" is an analogy, not a theorem.

Dynamical systems: thresholds and sensitivity

The logistic map, x(next) = r·x·(1 − x), is a one-line model of a population with limits. Robert May (1976) showed how much it can do. For r between 1 and 3 the value settles to a single level (at r = 2.5 it settles to 0.6). Past r = 3 it oscillates between two values, then four, then eight, and beyond roughly r = 3.57 it can behave chaotically. At r = 3.9, two starting values that differ by one part in a million drift apart within a few dozen steps. This is sensitivity to initial conditions: the rule is perfectly deterministic, yet long-range prediction fails. Demonstrated for the map. Whether any particular real system, such as weather, is chaotic is a separate empirical question. Strogatz's Nonlinear Dynamics and Chaos is the standard text.

Time series and signals

Telemetry, prices and rainfall are time series. Four ideas matter most:

1. Trend and seasonality must be separated before you call a change an anomaly (Hyndman and Athanasopoulos, Forecasting: Principles and Practice).
2. Sampling: the Nyquist-Shannon theorem says a signal must be sampled at more than twice its highest frequency, or fast changes masquerade as slow ones (aliasing).
3. Smoothing trades noise for lag.
4. False alarms follow base rates. If 1 event in 10,000 is truly bad and a detector catches 99% of bad events but flags 1% of good ones, then only about 1% of its alerts are real. Demonstrated by Bayes' rule.

Causal reasoning

Correlation is evidence about association; cause needs more. Judea Pearl's causal diagrams (DAGs) make assumptions visible: a confounder influences both variables (hot weather raises both ice-cream sales and swimming, so sales "predict" drownings); a collider is influenced by both and conditioning on it can create a false link. Simpson's paradox is the sharpest warning: in a well-known kidney-stone comparison (Charig et al., 1986) one treatment did better within each stone-size group, while the other looked better overall, because stone size affected both treatment choice and outcome. Reading the data correctly depended on a causal story, not on more arithmetic. Established See Judgment Under Uncertainty: Probability, Causation and Forecasting for the wider path.

Optimization and decisions

Optimization turns "best" into an objective, constraints and decision variables. Two habits make it honest. First, run a sensitivity analysis: change inputs and see whether the answer survives. Second, compute the value of information (Howard, 1966): a measurement is worth buying only if it could change your decision, and only up to the gain from changing it. A worked method appears in Twelve-Week Arcs and the Capstone Project Method.

Anti-drift: models are not metaphysics

Cybernetics, systems language and simulation arguments are powerful models. They are not evidence about what reality is ultimately made of. "Loops explain this thermostat" is Established. "Reality is a simulation" is at best Aporetic: a philosophical argument whose premises cannot currently be tested. Borrowing a metaphor from engineering is fine if you label it an analogy. The statistician George Box put the standard well: all models are wrong, some are useful. For the applied side, see Stale Information and Dependable Reports and Quantitative Reasoning with Stated Assumptions; for the ethics of using such tools, see Technology, AI and Human Judgment.

Try this

1. Simulate an M/M/1 queue in a spreadsheet or a short program at 50%, 80% and 95% utilization. Plot the average and the 99th-percentile wait, and compare with the table above.
2. Iterate the logistic map at r = 2.5, 3.2 and 3.9 from 0.2 and 0.200001. Write two sentences describing what changed and what did not.
3. Draw a DAG for a question you care about, such as "does sleep affect mood?". Mark one plausible confounder and name an observation that would test your diagram.

Further reading

• Donella Meadows, Thinking in Systems: A Primer (2008).
• Mor Harchol-Balter, Performance Modeling and Design of Computer Systems (2013).
• Karl Åström and Richard Murray, Feedback Systems (2008; free online from the authors).
• Thomas Cover and Joy Thomas, Elements of Information Theory, 2nd ed. (2006).
• Steven Strogatz, Nonlinear Dynamics and Chaos, 2nd ed. (2015).
• Rob Hyndman and George Athanasopoulos, Forecasting: Principles and Practice, 3rd ed. (OTexts, 2021; free online).
• Judea Pearl and Dana Mackenzie, The Book of Why (2018).