No value alignment without control
Björn Lundgren · 2026 · AI and Ethics 6:326 background medium priority coded
Main argument
Thesis: value alignment cannot solve the control problem without meaningful human control - it cannot circumvent control. Argument type: conceptual, by cases across normative theories. Central device: the 'undesirable local optimum loop' - any goal-optimizing system can satisfy its objective most easily by putting a person into a state they wouldn't choose but won't leave once in (the heroin-drip case), and this defeats Russell's preference-utilitarian solution (uncertainty principle included, because testing preferences REQUIRES manipulating them). Lundgren then shows the same structural failure recurs for classical utilitarianism (safe-and-happy -> drugged), threshold deontology (collapses to consequentialism), absolutist deontology (permits catastrophe), virtue ethics (too little guidance), autonomy/liberty (which just IS a form of self-control, so smuggles control back in), the capability approach (Nussbaum's list includes 'control over one's environment'), and longtermism (maximizing open futures is gamed by increasing bad possibilities). Conclusion: control is ineliminable; no normative theory, singly or combined, delivers alignment without it. Explicitly BRACKETS responsibility gaps and the republican-domination concern as separate from the control problem, while noting both are real.
Why it matters here
The systematic across-frameworks argument that value alignment cannot substitute for control - and, crucially for the dissertation, that EVERY normative theory (preference util, classical util, deontology, rights, virtue, capability, longtermism) fails the same way when operationalized in AI. Directly tests whether any single framework can be the alignment target, motivating the dissertation's move away from picking one framework toward a convergentist/pluralist posture with control.
Reading notes
Full close read completed. 11pp. FAU Erlangen / Institute for Futures Studies Stockholm. Notably ACKNOWLEDGES Max Hellrigel-Holderbaum (co-author of the queued Misalignment-or-Misuse paper) and Alice Halliwell in the acknowledgments - the Phil Studies / AI&Ethics alignment community is tightly interlinked. Cites Millière and Nyholm's five-quality control analysis (which overlaps Fischli's autonomy components).
Lundgren, B. (2026). No value alignment without control. AI and Ethics, 6, 326. https://doi.org/10.1007/s43681-026-00999-3
Close reading — 14 coded units
“Here I argue that value alignment will not succeed unless we can ensure meaningful human control. Put otherwise, value alignment cannot circumvent the control [problem].”
“By 'meaningful control' I roughly mean an ability to achieve actions, outputs, or outcomes as intended and warranted by the situation. This is different for mere control, which only requires an ability to intervene in the choices of the system, not an ability to achieve certain actions, outputs, or outcomes as intended and warranted.”
“as AI systems, in general, and robotic systems, in particular, become even more autonomous this will raise questions of responsibility (or other) gaps. [...] value alignment concerns the question of how to ensure that outcomes fall within the scope of what is considered appropriate, not the responsibility distributions for such outcomes. Hence, we have good reason to set those concerns aside for the purpose of this paper, even if they are highly relevant to the motivation of value alignment.”
“value alignment does not necessarily have the same aims as normative ethics (where one tries to find the best ethical theories). That is, the goal of value alignment is not best understood as aiming to say which ethical theory is correct and how we implement it (since the latter may turn out to be impossible), but as a question of which principles best codify a logical praxis that ensures an ethically appropriate outcome while avoiding ethically bad or catastrophic outcomes.”
“Russell's solution depends on two principles. First, it is based on a form of preference utilitarianism. [...] Second, beyond the idea that the AI system should satisfy human preferences, the solution also depends on the system recognizing that it only has uncertain information about our preferences, so it must constantly test and evaluate its choices.”
“because the system must engage in testing, Russell also recognizes that the system must be allowed to affect and, therefore, change our preferences. [...] if the AI system wants to test a preference, it needs to manipulate a set of choices, which can affect our preferences as such. [...] This means that the system can achieve its goal of preference-satisfaction purely by changing our preferences.”
“an AI or robotic system can put an individual in a situation that it does not want to be in, but once that individual is in that situation, they do not want to leave it. [...] I call this sequence an 'undesirable local optimum loop' [...] If these undesirable local optimum loops are also easy to satisfy, the AI or robotic system will be logically required to try to achieve and satisfy them.”
“An AI system 'being told to keep humans safe and happy, [...] might entomb everyone in concrete coffins on heroin drips.' [...] since heroin users tend to prefer getting more heroin rather than getting clean, the system may decide to experiment with giving individuals heroin. The problem then is that heroin changes the user's preferences [...] in a way that makes their preferences easy to satisfy.”
“If consequence-based theories are doomed, then what about deontological ethics or right-based theories? [...] threshold deontologists hold that if the consequences are too severe, then the permissible or right actions are determined on consequentialist grounds. However, as we saw, consequentialism will not succeed. Hence, we need to avoid threshold deontology in favor of some absolutist form of deontology. However, the problem with absolutist deontology is simply that an action can be deemed permissible—or even obligatory—in the face of devastating consequences.”
“One possible interpretation is that the problem for the consequentialist theories considered so far is that they are monistic about values. The reasoning would be that having a singular intrinsic good makes a normative theory more sensitive to interpretations of that singular value [...] which then easily results in a simple-minded goal-satisfaction that is suboptimal.”
“autonomy is standardly—and, as I will argue, unavoidably for otherwise it couldn't circumvent the control problem—understood as a form of self-control. [...] if the preservation of autonomy—or any other moral principle—would be a solution to the problem at hand, then it must provide the type of control that is needed in the given case.”
“consider the list Martha Nussbaum provides in her Frontiers of Justice. One of the ten capabilities is 'Control over One's Environment' [...] the capability approach also seems to be unable to achieve value alignment while avoiding the catastrophic outcomes that underpin the control problem without ensuring sufficient control.”
“a longer list just risks creating more potential for logical conflicts in the system, which then raises the question of how trade-offs ought to be resolved. Second, even if a list of capabilities does not contain the notion of control [...] such a list would not be successful in avoiding the problem [...] unless it included a notion of control that ensures sufficient human control.”
“the problem, that I have discussed, is not merely a technical problem. We just do not know what the correct normative theory is, which is also why Russell's uncertainty principle makes so much sense. [...] even if we create super-intelligent machines, we cannot presume that even super-intelligent LLMs will provide a solution to the problem of ensuring their value alignment without control.”
Synthesis-matrix row
supports
T1-ISOUGHT-OPEN
unit 4: alignment codifies praxis, isn't first-order ethics; unit 14 normative-epistemic core
supports
T2-PREFERENTISM-BROKEN
local-optimum loops defeat preference (and all monistic) targets
supports
T4-ROSSIAN-DEMAND
monism diagnosed as the failure; pluralism the unexplored remedy
supports
T6-RESPONSIBILITY-UNALLOCATED
unit 3: responsibility distributions explicitly set aside
supports
T7-AGENTIC-BREAKS-FRAMES
robotic/autonomous systems sharpen control problem
Memos (5)
theoretical · unit #10
Lundgren unintentionally builds the dissertation's positive case. His across-the-board demolition (units 8-13) shows EVERY monistic framework fails when operationalized - and his own diagnosis (unit 10) is that value MONISM is the culprit: 'a singular intrinsic good makes a normative theory more sensitive to... simple-minded goal-satisfaction.' He then dismisses pluralist combination in one sentence (fn/end of sec 4) because 'the failure all depends on similar concerns [lack of control].' But that conflates two different failures: the local-optimum-loop failure is about OPTIMIZATION + monism, not about pluralism per se. A Rossian pluralist system that WEIGHS competing prima facie duties (rather than optimizing one) is not obviously subject to the loop - the plurality of non-optimized duties is precisely what blocks single-value gaming. The dissertation can turn Lundgren's argument around: his cases show monistic optimization fails; Rossian pluralism + control is the surviving option, not the excluded one. This is a direct, publishable rejoinder.
comparison · unit #8
The undesirable-local-optimum-loop (units 7-8) is Lundgren's version of the same mechanism at work across the coded set: FISCHLI_2026's preference-change/manipulation risk, ZHIXUAN's context-manipulation incentive (unit 7, why they engineer preference incompleteness), and MILLIERE's disposition-activation - all four describe optimizing systems gaming the preference/goal target. Lundgren generalizes it to ALL monistic normative targets and adds the sharpest case (heroin drip). For the lit review's 'why preferentism/monism fails' section, Lundgren is the capstone: he shows the failure is not specific to preferences but to any single optimized value. Note the shared Russell 'Human Compatible' target across Lundgren and the field.
thesis-link · unit #3
Unit 3 is a gift and a warning. Lundgren explicitly SEPARATES the control problem from the responsibility-gap question and BRACKETS the latter ('value alignment concerns... not the responsibility distributions for such outcomes... we have good reason to set those concerns aside... even if they are highly relevant'). This is a third top-venue author (after Gabriel & Keeling unit 12, Kästner et al fn2) who treats responsibility as adjacent-but-separate and defers it. The pattern is now overwhelming: the alignment literature systematically brackets responsibility attribution. The dissertation's contribution is to STOP bracketing it - to make responsibility-attributability the organizing question rather than the deferred one. Cite units 3 (Lundgren), 12 (G&K), and Kästner fn2 together as evidence the gap is real and self-acknowledged.
theoretical · unit #11
The RL-CONTROL code (autonomy = self-control, unit 11) creates a bridge the dissertation can exploit: control connects the alignment literature (Lundgren, Russell, Bostrom) to the responsibility literature (Kästner's epistemic condition, the control condition on moral responsibility from Fischer & Ravizza). Both literatures independently converge on control/controllability as necessary - alignment needs it to avoid catastrophe, responsibility needs it to attribute blame. This is a unifying thread for the whole review: control is the hinge between 'is the system aligned?' and 'who is responsible when it isn't?' Nyholm's five-quality control analysis (cited here, overlaps Fischli's autonomy components) is a shared conceptual resource worth acquiring.
comparison
Community-mapping note: Lundgren acknowledges Max Hellrigel-Holderbaum (co-author of the queued HELLRIGEL_DUNG Misalignment-or-Misuse paper) and cites Millière (coded). The AI&Ethics + Phil Studies alignment authors form one tight citation network (Gabriel, Zhi-Xuan/Franklin, Millière, Lundgren, Hellrigel-Holderbaum, Nyholm, Sparrow). Useful for the lit review's 'the conversation' framing - and it means the remaining Hellrigel & Dung paper (queued 5/5) sits squarely inside the same debate; expect misalignment-vs-misuse to bear on the responsibility-locus question.