The Impossibility Theorem: You Cannot Vote Your Way to a Hierarchy of Values

In 1951, economist Kenneth Arrow identified a fundamental constraint on how democratic systems translate individual preferences into collective decisions: Arrow's Impossibility Theorem.

I first encountered Arrow's Impossibility Theorem in a political theory seminar almost 20 years ago. It stayed with me because it was uncomfortably concrete about the structural limits of collective decision-making. Last week, sitting in a working group at The World Forum on the Future of Democracy, Technology and Humankind in Berlin, the theorem came back to me - this time not as a question of political theory, but as a live governance problem.

The theorem, in short, constitutes that no voting system can satisfy a set of seemingly reasonable fairness conditions at the same time. At its core, it's a statement about the structure and limitations of collective decision-making.

Consider what those conditions imply:

• Non-dictatorship means that no single individual's preferences should automatically determine the group's decision.

• Pareto efficiency means that if every individual prefers option A over option B, the collective decision should reflect that. If everyone agrees, the outcome should match. A system that produces outcomes nobody wants - when a clear consensus exists - is failing at the most basic level.

• Independence of irrelevant alternatives means that the ranking of two options should depend only on how individuals rank those two - not on some third, unrelated option. This is the criterion that gets violated most often and most consequentially. It explains why the presence of a minor third-party candidate can shift outcomes between major ones - and why agenda-setting is itself a form of power.

• Unrestricted domain means the system should work for any possible set of preferences - not just convenient ones. A voting system that only functions when people's preferences happen to be arranged in a particular way is not a robust democratic mechanism.

Each of these conditions is intuitive. However, Arrow shows that together, they are mathematically incompatible. You cannot satisfy all four conditions simultaneously. That is Arrow's result: even under ideal conditions - coherent preferences and stable inputs - collective decision-making cannot be perfectly fair and consistent at the same time. Something has to give.

This didn't stop democracies from functioning. But it drew the boundaries within which they could operate.

Those limits are now becoming harder to ignore.

Modern democracy has been operating without ever fully deciding what it is optimizing for - and treating that ambiguity as a feature rather than a flaw. Arrow showed that no decision-making system can avoid trade-offs. And AI is now forcing those trade-offs into the open.

For decades, this was treated as a theoretical constraint rather than a practical design problem. Democratic systems functioned well enough that the tension could be managed in practice.

But the conditions that made that possible are now eroding.

The Question Arrow Forces Us to Ask

If no decision-making mechanism can satisfy all reasonable criteria at once, continuing to search for a better mechanism misses the point.

A harder question sits underneath: what should take precedence when these criteria conflict?

Arrow's theorem implies something that remains under-appreciated: democratic systems cannot derive their foundational values from the aggregation process itself. You cannot vote your way to a hierarchy of values - because the voting procedure already embeds one. Someone has to decide, in advance, what matters more when trade-offs arise: individual equality or outcome efficiency, freedom of expression or protection from manipulation, procedural neutrality or epistemic integrity.

These are not technical questions. They are political and philosophical ones. And for the most part, democratic systems have avoided answering them explicitly.

Why AI Changes the Stakes

In the pre-AI era, that avoidance was sustainable. Institutions operated within an implicit hierarchy of values, and the pace of change allowed tensions to be managed through negotiation, compromise, and ambiguity. But AI is changing that. It accelerates decision cycles, compresses time for deliberation, and forces trade-offs earlier and more frequently. At the same time, it destabilizes the informational environment in which preferences are formed.

This matters because Arrow's result assumes something important: that individual preferences are coherent, stable, and meaningfully comparable.

Those assumptions are now under pressure. Preferences are increasingly shaped by personalized information environments, algorithmic curation, and synthetic content. The "irrelevant alternatives" that Arrow warned about are no longer occasional distortions - they are a persistent feature of the system.

Therefore, the problem is no longer just how to aggregate preferences. The conditions under which those preferences are formed are becoming increasingly unstable. And subsequently, the limitations Arrow identified are no longer merely theoretical edge cases, but might become significant constraints of our democratic decision-making processes.

The Design Problem We Deferred

Arrow's theorem points to a requirement that has always been there, but rarely made explicit: a functioning democratic system needs a prior, explicit ordering of values - not as an output of collective decision-making, but as a precondition for it.

That means deciding, in advance and in principle, how conflicts between competing values are resolved - not leaving it to be negotiated case by case under time pressure or political constraint.

In the context of AI governance, this becomes concrete very quickly:

• When freedom of expression conflicts with protection from manipulation, which takes precedence?

• When innovation outpaces regulatory capacity, what principles guide intervention

• When national sovereignty diverges from collective standards, how are trade-offs resolved?

These questions are, in fact, currently being answered - implicitly, inconsistently, and often by default. Not through deliberate governance decisions, but as a byproduct of institutional design.

The Recursive Constraint

There is, however, a deeper difficulty.

Establishing a hierarchy of values requires deliberation. Not just consultation - genuine collective reasoning, in which participants can engage with each other's arguments, update their positions, and arrive at outcomes that carry democratic legitimacy.

That process is under pressure from two directions simultaneously:

The first is epistemic. Deliberation depends on a shared informational environment - not agreement, but enough common ground to make disagreement legible. To know what you are actually arguing about. AI is systematically eroding that ground. Personalized information environments mean that citizens increasingly inhabit separate epistemic worlds. Synthetic content makes it harder to distinguish what is real from what is manufactured. Algorithmic curation shapes not just what people see, but what they believe others believe - distorting the social perception on which deliberation depends.

The second is substantive - about the content of disagreement itself. Even where deliberation is possible, disagreement over foundational values is widening. Increasingly heterogeneous and globalized societies bring a broader range of value systems into direct contact - and into direct conflict. The baseline assumptions that once allowed democratic systems to function on implicit shared values can no longer be taken for granted. What counts as a legitimate trade-off, what rights are non-negotiable, and what the purpose of collective decision-making is - these are no longer settled questions, if they ever were.

Together, these pressures create a deadlock: governing AI requires a democratic values hierarchy. Establishing that hierarchy requires deliberation. Deliberation depends on both the epistemic conditions and a minimal level of shared values - both of which are now eroding.

You cannot reach the solution without passing through the conditions that are breaking down.

What Follows

The problem is not that democratic systems are making the wrong trade-offs. It is that they are making them without naming them - and therefore without being able to justify or contest them.

And equally, the task should not be to define a perfect decision-making mechanism. That does not exist.

Arrow's theorem makes something else clear: every system already makes trade-offs. It privileges certain principles over others - whether explicitly or not.

The difference is that most democratic systems have treated those trade-offs as implicit.

That is no longer viable.

If no system can satisfy all fairness conditions at once, then the task is to decide - explicitly and in advance - which ones matter most, and under what circumstances they can be overridden.

In practice, this means being explicit about tensions that cannot be resolved without loss:

• protecting freedom of expression may allow forms of manipulation that degrade collective decision-making

• constraining manipulation may require limiting certain types of speech or amplification

• maximizing participation may come at the cost of epistemic quality

• enforcing higher epistemic standards may exclude or silence certain actors

• enforcing common standards across jurisdictions may override what individual democracies have chosen - but without them, the result is fragmentation: incompatible frameworks, governance gaps, and no shared basis for accountability

None of these outcomes is neutral. Each reflects a prioritization.

In a world where these trade-offs are unavoidable, democratic legitimacy depends not on eliminating them - but on making them visible. On stating clearly which values are being prioritized, and why.

Arrow did not give us a way out of this problem. He instead showed us that it cannot be solved cleanly. What follows from that is not a single answer, but a requirement: that the trade-offs embedded in democratic systems be made explicit, rather than assumed or obscured.

So, the first step is not to resolve these tensions, but to surface them as design choices. Every regulatory threshold, every content moderation policy, every decision about what AI systems may optimize for already reflects a ranking of values. These are not neutral technical defaults, but political settlements that are often made without any deliberation. The task is to treat these embedded hierarchies as what they are: contestable decisions that require justification, not just implementation.

What are the thoughts that keep YOU up at night? Share your questions in the comments - how will AI reshape our international order? Your ideas will help set the agenda for upcoming articles.

If you found this valuable, share it with your network. I write more on emerging technologies, geopolitics, and international peace and security on my Substack: Thoughts That Keep You Up at Night