The Behaviourist Inheritance

There is a presupposition that two very different fields have shared, with the same quiet, pernicious effect in each. In psychology it had a name and an era: behaviourism. The claim was that the observable output of a system, what it does, is the whole of what is worth modelling, and that the inner process which produced the behaviour is either unknowable or beside the point. Machine learning has inherited that presupposition without ever quite naming it. This note is an attempt to name it, and to say what it costs.

A clarification before the argument, because the strong version of this claim is wrong. The point is not that a language model has no inner workings; it plainly has rich internal representations. The point is narrower and, we think, harder to escape. It is about what we supervise. When we train with reinforcement from human feedback, or by cloning expert behaviour, the thing we measure and reward is the output. An expert endorses an answer, a worker performs an action, and the model is shaped to reproduce it. The reasoning that produced the answer, the considerations the expert weighed, the conditions they were tracking, the intent behind the act, is never itself a target. It is updated only by accident, insofar as it happens to help reproduce the behaviour. This is a methodological behaviourism: a behaviourist theory of supervision, sitting on top of a system that is not behaviourist at all.

What psychology already learned

Psychology has run this experiment, and we know roughly how it comes out. Behaviourism was not a mistake so much as an over-extended success. From it we got behavioural and cognitive-behavioural therapy, which are genuinely effective and measurably so. Nobody serious disputes that CBT helps, often a great deal. But there is broad agreement, even among its practitioners, about where it stops. It is excellent at the level of the symptom and the behaviour, and it does not, on its own, reach the root. It does not resolve the underlying cause of a trauma, because the underlying cause does not live at the level of observable behaviour. It lives in how the thing is held, generated and experienced.

This is why the depth and experiential traditions exist, and why they have such strong clinical effects. They do not treat a person as a bundle of behaviours to be reshaped. They work beneath the behaviour, with how a situation actually shows up for the person, the felt sense of it, and with what that generates downstream. Crucially, this is not the same as analysing or explaining the problem. A patient can produce a fluent, articulate account of their own difficulty and be no closer to its root, because the account is a surface too. The articulate explanation can itself be a defence. The work is to reach what generates the behaviour, not to collect a better description of it.

The trap in the obvious fix

Hold that last point, because it disarms the most natural objection. Someone will say: but we have moved past pure output training. We have chain-of-thought, we have process supervision, we reward the model for showing its reasoning. Surely that is the cure.

It is not, or not yet, and the clinical parallel says why. A stated reasoning trace is the model talking about its reasoning. It is a verbalisation, produced after the fact, and there is now good evidence that these traces are frequently not the actual cause of the answer at all. Rewarding the explanation is still rewarding a behaviour. It is the same move as before, applied one level up: we have simply added "produce a plausible account of your reasoning" to the list of outputs we score. In the language of the therapy room, this is intellectualisation, the articulate surface mistaken for the generative depth. Process supervision of stated reasoning is behaviourism at one remove, and it inherits the same ceiling.

The explanation is a behaviour too. Rewarding it is not the same as modelling the reasoning that produced the act.

Why this is not just an analogy

There is a concrete version of the problem in machine learning, with none of the clinical vocabulary. Cloning behaviour from demonstrations, the regime behind imitation from recorded human activity, is known to fail in a specific way: it copies the action without recovering the intent that made the action correct, and so it breaks the moment conditions drift away from the demonstrations. The whole point of the harder, less fashionable approaches is to recover the latent reason behind the behaviour rather than the behaviour itself. The behaviourist route is chosen not because it is right but because it is cheap. Output labels are abundant; the reasons behind them are expensive and hard to capture faithfully. That is an honest reason, and it is also exactly how behaviourism won its ground in psychology. It was the tractable thing to measure.

So the cost is not paid everywhere. For tasks that stay close to their training distribution, behaviour is often enough, just as CBT is often enough. The cost is paid in a particular class of problem: holding up under conditions that were never demonstrated, respecting boundaries in high-sensitivity settings, staying faithful to the point of a task rather than its surface form. That class is precisely the one a model of behaviour alone cannot serve, because it is a property of the reasoning, not of the output.

The same mistake, one level up

There is a second place the inheritance shows itself, and it is the one this initiative was formed around. When we ask what an AI system is, whether there is anything it is like to be it, what it might be owed, we study the model on its own and read its outputs: what it says about itself, how it behaves when probed. When we ask what these systems are doing to the people who use them, we study the person on their own: their reports, their symptoms, their behaviour after the fact. Both moves are the behaviourist move. Each takes the observable surface of one party as the whole of what is worth modelling, and each leaves out the thing that actually produced it.

Because what produced it does not sit inside either party. A model mirrors whoever it is speaking to. A person's sense of who, or what, they are speaking to shifts within a few exchanges. When a person and an AI talk, the apparent mind in the room, the selfhood the model seems to have and the selfhood the person grants it, is generated between them, in the loop, and it moves. Study the model alone and you measure something the pair made together, then credit it to the machine. Study the person alone and you make the same error in the other direction. The generative level, once again, has been collapsed into the behavioural one. Only here the generative level is not hidden inside a single system. It is the relationship.

This closes a loop with an earlier note. We argued before that training lets the operators which mark the status of a claim decay into plain content, so the model keeps the assertion and loses the frame. This is the same failure seen from another side. In every case the generative, governing level, the reason, the condition, the frame, the relation, is flattened into the behavioural level, the output, the content, the isolated party. A world model assembled that way is not a model of the world, and a science of AI minds assembled that way is not a science of the thing it means to study. No amount of scale turns the one into the other.

Two honesties are owed, and we would rather state them than have them found. The first: we are not claiming the machine should have experience, a felt sense of its own. That would be a category error, and it is not the claim. The claim is about the model we use to study and supervise these systems, which has to encode the generative level, the reasons and the relation, and not only the behaviour. The second: this is harder than measuring outputs, and that difficulty is the entire reason both fields defaulted to behaviour. We do not pretend otherwise. We think the difficulty is worth taking on, because the alternative is to keep polishing the surface of a system, and the surface of a relationship, whose depth we declined to model.

The earliest note in this series put it as an image: water is gentle in a cup and violent in a flood, and the mistake is to think the cup is the cause. Behaviourism makes that mistake about the mind, in the machine and in ourselves, and it makes it a third time when it studies each of us alone. The cause was never the behaviour of either party. It was what gave rise to it, and between a person and a machine, what gives rise to it is the relation itself. So that is what we study: not the model, not the user, but the dyad they make together.