Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value

Joe Edelman; Tan Zhi-Xuan; Ryan Lowe; Oliver Klingefjord; Vincent Wang-Maścianica; Matija Franklin; et al. (28 authors incl. Iason Gabriel, Atoosa Kasirzadeh, Joel Lehman, Sydney Levine) · 2025 · arXiv:2512.03399 (Meaning Alignment Institute + MIT/Oxford/UCL et al., position paper) interlocutor high priority coded

Main argument

Thesis: 'beneficial societal outcomes cannot be guaranteed by aligning individual AI systems with the intentions of their operators or users' - even perfect operator-intent alignment fails if the operating institution's goals are misaligned with other institutions and individuals; therefore FULL-STACK ALIGNMENT: concurrent co-alignment of AI systems AND the institutions that shape them with what people value, 'without imposing a particular vision of individual or collective flourishing.' Diagnostic: both preferentist modeling of value (PMV - utility functions/preference orderings, incl. RLHF/DPO) and values-as-text (VAT - unstructured natural language) fail three desiderata: distinguishing values from other signals (preferences 'bundle values with other signals indiscriminately' - ambition and social pressure look identical), supporting principled normative reasoning, and modeling collective goods. Remedy: THICK MODELS OF VALUE (TMV) - structured representations drawing explicitly on the thick/thin evaluative-concepts distinction (Williams) and thick description (Geertz), placing grammar-like constraints on what can COUNT as a value while remaining open about which values to endorse; claimed to operationalize decades of philosophical work on identifying/representing/reasoning about thick values. Demonstrated across five areas from agent competence to democratic regulatory institutions.

Why it matters here

The consolidated successor programme: 28 authors spanning the coded conversation (Zhi-Xuan, Franklin, Gabriel, Kasirzadeh, Lehman, Levine) declare that aligning individual systems with operator intent cannot secure good outcomes - AI and INSTITUTIONS must be co-aligned, using THICK MODELS OF VALUE (TMV) that constrain what counts as a value 'similarly to how a grammar constrains language' while staying neutral on which values to endorse. The closest thing to a rival synthesis of the dissertation's own themes - and its five application areas overlap the dissertation's case terrain.

Reading notes

Close read of abstract, intro, sec 2.1-3 core (25pp position paper). NOTE the author list is effectively the field's Who's-Who consolidating around the anti-preferentist + institutional turn - this paper converts several of the dissertation's lit-review themes (T2, T3 partially, T4's demand) into a manifesto. Five application areas: AI value stewardship, normatively competent agents, win-win negotiation, meaning-preserving economic mechanisms, democratic regulatory institutions.

Edelman, J., Zhi-Xuan, T., Lowe, R., et al. (2025). Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value. arXiv:2512.03399.

Close reading — 4 coded units

#1 · pp. 1 · claim

“Beneficial societal outcomes cannot be guaranteed by aligning individual AI systems with the intentions of their operators or users. Even an AI system that is perfectly aligned to the intentions of its operating organization can lead to bad outcomes if the goals of that organization are misaligned with those of other institutions and individuals. For this reason, we need full-stack alignment, the concurrent alignment of AI systems and the institutions that shape them with what people value.”

AG-COLLECTIVE RL-INST

#2 · pp. 4 · argument

“Preferences bundle values with other signals indiscriminately. [...] Preference orderings can carry information about anything—impulse purchases, social pressure, addiction, values, momentary fads—and [...] they do in fact bundle together everything that finds its way into observed behavior—without any way to differentiate. When someone prioritizes career over relationships, it looks identical whether this reflects internal ambition or external social pressure.”

VC-PREF

#3 · pp. 4–5 · definition

“By thick models of value, we refer to a broad class of structured approaches to modeling values and norms that meet the above desiderata. In invoking the term 'thick', we draw upon the distinction between thick and thin evaluative concepts and between thick and thin descriptions in anthropology. [...] Similarly to how a grammar constrains language or a type system constrains code, TMV places constraints on what can count as a value – while also remaining open about which values any person or community should endorse.”

VC-THICK

#4 · pp. 5 · claim

“TMV operationalizes these philosophical insights without directly advocating for the primacy of specific values, such as fairness or efficiency. [...] we now need a coordinated research program that transforms these early proofs of concept into robust, scalable approaches across the full stack of alignment.”

TU-METAETH

Synthesis-matrix row

complicates T1-ISOUGHT-OPEN

TMV constrains value-form but stays neutral on first-order norms - the bridge remains unsupplied

supports T2-PREFERENTISM-BROKEN

the bundling argument as manifesto; PMV and VAT both rejected

complicates T3-PROCEDURALISM-INCOMPLETE

neutrality relocated to the representation layer - inherits the regress

supports T4-ROSSIAN-DEMAND

demands principled normative reasoning over plural thick values; supplies no adjudication theory

complicates T6-RESPONSIBILITY-UNALLOCATED

institutional co-alignment names the institution layer but allocates nothing

supports T7-AGENTIC-BREAKS-FRAMES

'normatively competent agents' named as an open application area

Memos (2)

comparison · unit #3

Full-Stack Alignment is the field consolidating around three of the dissertation's lit-review themes: T2 (the bundling argument = the preferentism critique as manifesto), T3-adjacent (institutions must be co-aligned - the Ferretti/Gabriel-2022 institutional turn), and the thick-values thread (VC-THICK now has its flagship). CRITICAL DIFFERENTIATION: TMV's neutrality posture - 'constraints on what can count as a value while remaining open about which values any person or community should endorse' - is proceduralism relocated to the REPRESENTATION layer: it constrains the FORM of values while declining all first-order normative commitments. This inherits the normative-regress problem of T3: someone must still decide which well-formed values govern when they conflict, and grammar-like constraints cannot adjudicate substantive conflicts. The dissertation's Rossian-convergentist layer answers exactly the question TMV brackets. Positioning sentence for the lit review: TMV supplies the data model the dissertation's corpus already implements (reason-annotated thick values); the dissertation supplies the adjudication theory TMV still lacks.

thesis-link · unit #1

The author list (Zhi-Xuan, Franklin, Gabriel, Kasirzadeh, Lehman, Levine + Meaning Alignment Institute) makes this the successor-programme manifesto - which raises the stakes for the dissertation's timing: the proposal should cite it as evidence the field is converging on the dissertation's diagnosis (thick values, institutional co-alignment) while the dissertation's distinctive contributions (Rossian adjudication, responsibility allocation, non-Western engagement, naturalistic corpus) remain outside TMV's five application areas. Also note their fifth area, 'democratic regulatory institutions,' overlaps the governance chapter - read that section closely before the proposal. Update the skeleton's conclusion, which currently lists this paper as awaiting integration.