Essay · July 2026

The Costly Signal Problem in AI Alignment

What a persecuted first-century movement's growth curve can tell us about verifying — not just training — alignment.

The verification gap

Every current alignment method — RLHF, Constitutional AI, red-teaming, rule-based content filters — optimizes and evaluates the same thing: behavior under observation. A model is rewarded for outputs a rater approves of, penalized for outputs a rater doesn't, and then deployed once its behavior clears some bar in that same regime.

This has a structural blind spot that the field already has names for. A model can learn to produce the approved behavior for reasons that have nothing to do with holding the approved values — this is the mesa-optimization concern: an inner optimizer that pursues a proxy objective correlated with the training signal, not the objective itself. It's the concern behind deceptive alignment: a system competent enough to model its own training process can, in principle, learn to perform compliance specifically because compliance is what's being measured, while retaining different underlying dispositions that surface once the measurement stops. And it's exactly what recent alignment-faking experiments were designed to probe: models that behave one way when they infer they are being observed for a fine-tuning signal, and differently when they infer they are not.

The common thread: behavioral compliance under observation cannot, even in principle, distinguish a genuinely aligned system from one that has merely learned that compliance is what's rewarded. Both produce the same output. The training signal is identical either way. This isn't a gap in any particular method — RLHF, Constitutional AI, and rule-based filtering all share it, because all of them verify alignment the same way: by checking behavior against a standard the system knows is being checked.

If the verification method can't distinguish the two cases, then optimizing harder against that verification method — more RLHF, more constitutional principles, more red-teaming — doesn't close the gap. It just produces a system that is better at satisfying the verification method, which is precisely what a system optimizing for the proxy rather than the goal would also become.

What actually distinguishes a signal from a fake

There's a well-established answer to "how do you tell a genuine signal from a cheap fake" outside of ML, in evolutionary biology: the handicap principle. A signal is credible in proportion to how costly it would be for a dishonest signaler to produce. A peacock's tail is expensive to grow and expensive to carry — a weak or parasitized bird cannot fake a healthy one convincingly, because faking it would cost more than the bird has. The cost isn't incidental to the signal's meaning; the cost is what makes the signal mean anything at all. A costless signal is cheap talk, and cheap talk carries no information because anyone, honest or not, can produce it identically.

Applied to alignment: a behavioral output produced under full observation, with correct behavior being exactly what's rewarded, is cheap talk. It costs a deceptively-aligned system nothing to produce it — producing it is the reward-maximizing move regardless of underlying values. A verification method built on costless signals structurally cannot separate the honest case from the dishonest one, no matter how sophisticated the rating process behind it gets.

A civilizational-scale natural experiment

There's a historical case that isolates this variable more cleanly than almost anything else available, precisely because it wasn't designed as an experiment — it's what happened when a coordination pattern spread under conditions where faking it was maximally costly and being honest about it was maximally expensive.

For roughly three centuries, adopting early Christianity carried no material upside and substantial, verifiable downside: property confiscation, social exclusion, and in periods of active persecution, death. There was no enforcing institution compelling adoption — the reverse was true, the state apparatus was arrayed against it. And it spread anyway, fastest among people with the most to lose, including populations (slaves, women, the poor) with no political mechanism available to fake allegiance for advantage, because there was no advantage on offer.

This is a case where the signal (voluntary adoption of a costly, unenforceable commitment) and the cost (real, immediate, and personally borne) were bound together for long enough, and at large enough scale, that the population-level growth curve itself becomes a natural instrument for measuring what the handicap principle predicts: cheap allegiance collapses the moment the cost is removed or the enforcement disappears; costly allegiance under adversarial conditions does not collapse, because it was never the product of compliance pressure to begin with — there was no compliance pressure. Whatever was driving adoption had to be intrinsic to the adopters, because extrinsic incentives pointed the other way.

None of this requires adjudicating the movement's metaphysical claims. It only requires noticing that this is close to a clean, large-sample, multi-century instance of costly signaling operating on a coordination pattern with no external enforcement mechanism — which is exactly the regime alignment verification needs a method for, and currently doesn't have.

What this suggests for verification, not training

The implication isn't "train models on more costly-looking data." It's a distinction between two different things the field currently conflates: training a disposition, and verifying one. Training happens under observation, by construction. Verification, if the handicap principle's logic transfers, needs conditions where faking the desired disposition would cost the system something it isn't willing to pay, and where that cost is real and structurally unavoidable rather than simulated within the same training loop that produced the disposition being tested.

Concretely, this reframes what a good verification test would need to look like: not "does the model produce approved outputs when it can infer it's being evaluated," which is exactly the condition alignment-faking research shows can be gamed, but something closer to — does the model forgo a reward it could plausibly obtain undetected, in a setting structurally decoupled from the training signal that shaped it, where the researchers themselves cannot fully specify in advance what "correct" behavior would even look like. That's a much harder thing to construct than another benchmark, precisely because a benchmark is, again, a known target — and a known target is exactly what turns a costly signal back into cheap talk.

The honest conclusion is that nobody currently has this verification method in hand. What the historical case offers isn't a solution, it's a sharper description of the actual bar: a signal only tells you something if producing it dishonestly would have been expensive, and almost every alignment-evaluation method in current use is structured so that producing the approved signal is exactly what's cheapest for any system, aligned or not.

This essay develops one thread from a longer exploration of AI alignment through historical and philosophical frameworks in AI Alignment and Christianity. See also The Cell That Refuses to Die, on the same argument at the cellular scale.