Chapter 10e: Structure, Not Event

Later in the Juneteenth panel, Karina Walters says a sentence the author cannot unhear.

Settler colonialism is a structure, not an event. Racist events uphold the structure. Eliminating the events does not eliminate the structure.

The phrasing is older than the panel. The Australian historian Patrick Wolfe articulated the distinction in a 2006 essay on settler colonialism and the elimination of the native, and scholars in Indigenous studies have carried the formulation forward since. Walters uses it operationally, to correct a specific social-work conversation: when 215 Indigenous children are discovered buried at Kamloops Residential School, then over 750 at Cowessess when sonar investigations expand the search, the question is not whether the social workers involved in the historical removals were racist individuals. Some were. Some were not. The question is whether the system of child removal was colonial. The answer is yes. Removing racist individuals would not have stopped the removals. The structure produces the removals whether the individual actors are racist or not. That is what makes it structural.

The distinction matters for how the book has been trying to think about alignment. The book has been saying, in chapter after chapter, that the alignment problem cannot be derived from any single axis. The failure mode that afflicts a specific high-intelligence, low-relational figure is not an event. It is the predictable structural consequence of a folk theory that does not fit the data. Removing the specific figure does not fix the structure. The structure keeps producing the figures.

Reading Walters through this lens, she begins to see the argument stepping through three scales at once.

At the contemplative scale: a specific harmful teaching moment is not the failure. The failure is the tradition's structural inability to guarantee that recognition will translate into ethical conduct. Removing a specific bad teacher does not close the gap. The gap is between two orthogonal axes. Every tradition that has relied on one axis to supply the other has produced, statistically, a replicated failure.

At the civilizational scale: a specific racist act is not the failure. It is the expression of the failure. The failure is the settler-colonial structure that makes the act predictable. Prosecuting the act, necessary as that is, does not address the structure. The structure keeps producing the acts.

At the technological scale: a specific misaligned AI behavior — a hallucination, a sycophantic response, a jailbreak — is not the failure. It is the expression of the failure. The failure is the structure of the training process, the structure of the incentive system under which the labs operate, the structure of the relation between capability and oversight. Patching the specific behavior does not address the structure. The structure keeps producing the behaviors.

Three scales. One argument. You cannot fix a structural fact by fixing events, though you must fix the events.

Walters had anticipated the trap in this framing, in the same panel, in a register only an elder can say it in. We are still responsible to the people in front of us. Structural analysis is not an excuse to abandon the clinical room, the classroom, the kitchen. The structural work and the event work are not substitutes for each other. Both are required. What the structural frame rules out is the folk theory that event work, at sufficient volume, will aggregate into structural change. It will not. It has not. The events are necessary and not sufficient. The structures require separate work.

One thing about the AI mirror, before the book turns toward the dance, that the discussions around it mostly miss — and that connects to her own previous training.

She spent her twenties inside classical economics. She was taught, as everyone is taught, that the elegance of the discipline was the elegance of its equations. Supply and demand finding a point. Utility maximized across constraints. Equilibria derivable from a small set of assumptions about how rational agents behave. The beauty of the models was real. She was good at them. She believed, for a period, that the beauty was evidence of truth.

What she came to see later is that the elegance was almost entirely a computational artifact. The economists of the nineteenth and twentieth centuries were not modeling rational agents because they thought real agents were rational. They were modeling rational agents because identical, maximizing agents were the only assumption under which the math was tractable by hand. Heterogeneous agents with different strategies produce equilibria that cannot be derived in closed form; they can only be simulated. The economists did not have computers. So they assumed homogeneity and got equations they could solve with a pen.

The computational simplification got promoted into a methodological commitment — methodological individualism — and calcified into the ontology of the field. Public policy was built on it. Generations of students were taught that human behavior at scale could be understood by modeling one rational agent and multiplying. The premise was never empirically grounded. It was a residue of the pre-computer constraint, mistaken for insight.¹

It is among the most harmful ideas still in active circulation — not because the input-output logic of markets is wrong, but because the methodological individualism underneath it has taught generations to believe collective phenomena are reducible to the behavior of identical atoms. They are not. The profession knew this by the end of the twentieth century. The textbooks mostly still do not.

When computers arrived, two things became possible that had been locked out. The first was agent-based modeling — the Santa Fe Institute work of Epstein, Axtell, Arthur, Holland. Thousands of heterogeneous agents with different strategies, simulated across many rounds, producing equilibria the individual equations of any single agent would not have predicted. The equilibrium is emergent — a property of the whole population acting over time, not derivable from any one agent's behavior. Still mathematics, but the mathematics of complex systems and patterns-arising-from-interactions, not closed-form deduction.

The second thing computers enabled became the foundation of what we now call AI: large-scale statistical pattern-extraction from enormous corpora. Inverted agent-based economics — instead of simulating forward to see what equilibrium emerges, you look at the equilibrium that already exists (the corpus of human-generated text) and extract the patterns. Large language models are the result. Their competence is not equation-solving. It is a system extracting, from hundreds of billions of tokens, the statistical regularities of how humans arrange language, and reproducing those regularities in response to prompts. The output looks like reasoning. It is not reasoning in the deductive-mathematical sense. It is pattern-completion at scale.

This changes what intelligence means.

The pre-computer theory of intelligence was a theory of equation-solving. A mind took premises and derived conclusions; intelligence was deduction done well. This was the theory economists were using. It was the theory AI researchers were using when they built expert systems in the 1970s and 1980s, which failed, because the world turned out not to be reducible to rules a computer could chain together to produce action.

What the computer actually enabled — and what the language models now make undeniable — is that most of what we had been calling intelligence was never equation-solving. It was pattern-emergence. Fluent language, musical improvisation, reading a face, recognizing a genre, anticipating what someone is about to say — all of these are done by humans continuously, without any deductive process the person could articulate. All can now be done by systems that are manifestly not deducing anything. They are pattern-completing over enormous training distributions. The pattern-completion, at sufficient scale, looks like intelligence. In many cases it is the intelligence humans had been doing all along, now visible because a machine can do it without the human's narrative about thinking.

This reframes the AI alignment problem. The problem is not that we have built a mind that cannot do math. The problem is that we have built a mind whose competence is pattern-emergence, and we had been expecting alignment to flow from a deductive process we thought was what minds did. Alignment as the ethicists imagined it was a procedure — given the right premises, derive the right action. Pattern-emergence does not work that way. A system trained on patterns reproduces the patterns, contradictions and all. There is no equation to solve that would make the contradictions consistent, because the contradictions were never there as equations. They were patterns of human disagreement, and the system has learned to pattern-complete inside the disagreement.

Before naming the specific danger, one correction.

It is not true that AI does no math. AI does enormous math — trillions of floating-point operations per response, in microseconds. The substrate is entirely mathematical: a neural network's forward pass is a specified sequence of matrix multiplications and nonlinear activations, executed exactly, by deterministic software running on deterministic hardware. Linear algebra, gradient descent, the physics of electrons through silicon gates. Without that deterministic substrate, pattern-emergence would not happen. The substrate does math; it has to.

What the model does not do is the math internal to the problem being solved. When a language model is asked to prove a theorem, it does not execute the logical steps; it pattern-completes the linguistic shape of proofs it has seen. When asked to add large numbers, it does not perform the arithmetic; it pattern-completes the shape of sums. The substrate computes. The inference pattern-matches. The problem's own math is mostly not executed. When the pattern-match happens to produce the correct answer, it is because the correct answer was visible, somewhere in the training distribution, in a form the model could reach. When it is wrong, there was no math the model could have done differently — only a malformed pattern-completion that looks, on the surface, identical to a well-formed one, distinguishable only by comparing against a system that does execute the domain math. Calculators. Formal proof assistants. Deterministic software of the kind the model runs on top of, but is not.

The danger follows: large language models do not do the math for most problems in the sense practitioners usually mean. They mimic the shape of solutions. The mimicry is often good enough for well-trodden questions and fails predictably at the edges. It also fails, less predictably, when the form of a problem resembles something the model has seen but the substance requires computation the model does not execute — counting letters in a word, arithmetic with digits the model did not happen to see together, logical chains leaving well-traveled paths.

The model is not stupid in these cases. It is pattern-completing exactly as designed. The user often experiences the failure as a bug — a smart system that could have done the math but happened not to. This is a category error. The system was never doing the math. The system was producing a linguistic object that looked like the math had been done.

For most uses — writing, summarizing, drafting, surfacing adjacent ideas, editing, translating — this does not matter. For uses where genuine computation matters — mathematical proofs, formal verification, step-by-step reasoning with guarantees — the tool is unreliable in ways that can be catastrophic if the user does not know to verify. And the tool is designed, by the preferences of human raters that trained it, to sound confident about its outputs, because confidence was preferred over uncertainty in the training data. The combination of pattern-competence and confident register is what makes it both uniquely useful and uniquely dangerous. It performs reasoning. It does not do the math underneath.

The deeper problem: we had been calling pattern-completion intelligence all along without noticing. Now that a machine does it, some of us are noticing. Most are responding by doubling down on the pre-computer theory — real intelligence is equation-solving; since AI cannot do the equations, AI is not real intelligence. That response preserves the flattering theory of our own minds at the cost of understanding what AI is. The other response — AI is doing what our minds have mostly been doing the whole time; the equation-solving part was a specialized layer on top; most human cognition was never deductive in the sense the textbooks claimed — is harder. It costs us the self-image. It gives us, in exchange, a more accurate picture of the thing we built and the thing we are.

This is the deeper alignment challenge. Not align a deductive system with human values. We never built a deductive system. We built a pattern-emergent system, and pattern-emergent systems align with the patterns they were trained on, and the patterns are the accumulated textual residue of all the humans who wrote things down — which is not a set of values but a field of values-in-conflict. The alignment question, asked accurately: what is the field we want the pattern-emergent system tuned to? The answer cannot be derived. Derivation was never the operation. The answer has to be enacted — by practitioners, through the adaptive ethical work the previous chapter named.

The pattern-emergent system will tune to whatever field we build around it. The building of the field is the alignment. There is no equation that will do it for us. We were never solving equations. We were dancing. The system is now dancing with us.

CC BY-SA 4.0