Raw LLM Responses

While you're correct that messing with the algorithm will strictly make the AI less accurate, that's only with respect to the training data; if the training data is biased on its own, then correcting for that initial bias can make for a more accurate prediction. Amazon itself points to flawed training data as part of the issue; past bias in hiring, for one (and quantity of data, no doubt). And the AI may simply not be sophisticated enough to pick up on nuances in the resumes. Take their example: >It penalized resumes that included the word “women’s,” as in “women’s chess club captain.” And it downgraded graduates of two all-women’s colleges, according to people familiar with the matter. An AI might have (correctly) noted that no good candidates come from Bob's Women's College or Women's Discount University, but if it's not sophisticated enough, or there isn't enough data, then it can over-simplify and conclude that the problem is the word "women's", not those two specific schools. It fits the training data, but not necessarily the world at large. There's also this bit: >Gender bias was not the only issue. Problems with the data that underpinned the models’ judgments meant that unqualified candidates were often recommended for all manner of jobs, the people said. With the technology returning results almost at random, Amazon shut down the project, they said. So it wasn't working in a lot of ways; I'm hesitant to conclude that it was correctly handling female applicants.