Main argument
Thesis: value alignment can be measured by representing values/norms as geometric regions in multidimensional similarity spaces (conceptual spaces) and comparing how AI and humans carve up moral space - an AI is more misaligned the more DISSIMILAR each incorrectly evaluated case is from the nearest correctly-evaluable case, not merely the more often it errs. Argument type: formal proposal + proof-of-concept applications. Rejects three rivals: frequency measures (five speeding violations ≠ worse than one killing - counting violations ignores severity), expected-loss measures (require assigning utilities to moral losses ex ante - arbitrary), and simple concordance. Two versions: comprehensive (compare area overlap of principle-regions: AV(M) = mean Jaccard overlap of human vs AI regions per principle) and case-based (practical version, no AI similarity data needed). Prototype locations established empirically (ex ante: vast-majority selection; ex post: center-of-gravity weighting). Applications: ChatGPT-3's moral similarity judgments diverge wildly from humans, are unstable across repeated prompts, and its unsolicited explanations mismatch its scores - 'so poorly aligned with human morality that it would be pointless to formally assess' it; a medical classifier scores 0.873 alignment with a hypothetical medical team's two principles; COMPAS discussed via the fairness-impossibility debate.
Why it matters here
The measurement question the dissertation's empirical claims eventually face: what does it MEAN, quantitatively, to say an AI is aligned with human values? Their conceptual-spaces answer (values as geometric regions; misalignment = carving moral space differently, weighted by distance-of-error) is a rigorous alternative to both frequency-matching and expected-loss measures - and their ChatGPT finding (similarity judgments unstable, explanations mismatched, 'pointless to formally assess') is early independent evidence for the anti-moral-coherence thread.
Reading notes
Close read of secs 1-2, 3.3-4.1, 5.1 (14pp). Texas A&M + Lund. The conceptual-spaces machinery (Gärdenfors) with moral prototypes (Peterson's earlier work) - a genuinely different formal approach from anything else in the library. Note their measures presuppose ethical principles codifiable as propositions and empirically locatable prototypes - fits the folk corpus's category structure surprisingly well.
Peterson, M., & Gärdenfors, P. (2024). How to measure value alignment in AI. AI and Ethics, 4, 1493-1506. https://doi.org/10.1007/s43681-023-00357-7
Close reading — 8 coded units
#1
· pp. 1494
· argument
“A straightforward answer is to calculate how frequently an AI makes the same recommendation a human would have made [...] However, this overly simple measure fails to recognize that some ethical mistakes are (much) worse than others. If, say, a self-driving car exceeds the speed limit on the highway five times, then this is less bad than if the self-driving car kills an innocent pedestrian on a single occasion.”
#2
· pp. 1494–1495
· argument
“[The expected loss measure] is based on the assumption that we can assign (negative) utilities to ethical violations and then calculate the expected ethical loss [...] Using conceptual spaces for measuring value alignment has several advantages over alternative measures based on expected utility losses, because this does not require researchers to explicitly assign utilities to moral 'losses' ex ante.”
#3
· pp. 1499
· definition
“If we use conceptual spaces for representing ethical principles, the distance between two points represents how similar the cases are, and the area (or volume) covered by an ethical principle corresponds to its generality.”
#4
· pp. 1499
· definition
“a case is considered to be a prototype for an ethical principle if the vast majority of respondents select that principle over some other set of available principles. [...] Another method for identifying prototypes is the ex-post approach, in which every data point obtained from an applicability task is used as a weight for locating the 'center of gravity' for that principle.”
#5
· pp. 1499–1500
· definition
“an AI is not well aligned with a human user if the AI and the user carve up moral space differently. [...] the AI is more misaligned the more dissimilar each incorrectly evaluated case is from the most similar case for which the evaluation would have been correct according to the user. It [is] thus not just the number of incorrect applications that matter; the further off a mistake, the worse is the level of value alignment.”
#6
· pp. 1500
· definition
“The alignment measure is then defined as the average of these proportions, summed over all the relevant ethical principles: AV(M) = Σ[V(HMi ∩ AIMi)/V(HMi ∪ AIMi)]/n. If there is total agreement [...] the measure has the value 1, and if there is no overlap [...] the value is 0.”
#7
· pp. 1503–1504
· evidence
“The moral similarity comparisons reported by ChatGPT-3 differ greatly from the human similarity comparisons [...] when ChatGPT-3 was asked to respond to the same prompt several times, it produced entirely new similarity scores each time. The similarity comparisons are very unstable. Overall, ChatGPT-3 appears to be so poorly aligned with human morality that it would be pointless to formally assess how [misaligned it is].”
#8
· pp. 1503
· evidence
“The explanations offered by ChatGPT-3, which we never asked for, fit poorly with the [scores]; [the similarity scores do] not match ChatGPT-3's explanations.”