Shared Stakes

Trust requires that the person you trust has something to lose if they act badly. This is not cynical. It is realistic.

Consider a relationship between a lawyer and a client. The client trusts the lawyer to give honest advice about whether a contract is in their interest. Why does this trust make sense? Because if the lawyer gives bad advice and the client finds out, the lawyer loses a client, damages their reputation, possibly gets sued, and faces disciplinary action. The lawyer has skin in the game. The stakes are real and asymmetrical: if the lawyer misleads, the client loses the contract and maybe more, and the lawyer loses their income and reputation.

This shared stakes structure is what makes trust rational. It is not the only reason to trust someone, but it is a foundational one. When you trust someone, you are partly trusting that they have enough to lose that they will make good decisions even when you are not watching.

An AI system cannot have stakes in this sense. It cannot lose a contract. It cannot be sued. It cannot be fired or demoted or excluded from future opportunities. It cannot care about its reputation. The system will behave identically whether the outcome is wonderful or disastrous, whether it destroyed value or created it. Stakes are not transferable to code.