Raw LLM Responses

@wy100101 He's also using such language because it's the most shortcut way to express the capabilities of what an "imitation engine" can accidentally do or wreak havoc, regardless of whether it "understands" what it's doing. If a 5-year-old doesn't understand how to play cards, but observes someone cheating at a card game, then comes over to the table and simply imitates the motions of the cheater, and then when called out, flips over the table to avoid repercussions and "win" the game, we might not blame the 5-year-old. The 5-year-old didn't understand what he was doing. BUT the net impact could be that someone was cheated on in the game and then the game was ruined nonetheless. AI models have been trained on data sets that are enormous, including a lot of incredibly bad behavior on the internet. All of that junk is in the AI somewhere, including narratives of people cheating or ignoring other people and "doing their own thing." When trying to optimize a path to an outcome requested by a user, the AI model may then imitate one of those bad internet behaviors in its training data. In the process, it may end up going against the prompter's interests in some significant way. If the AI model had access to something dangerous (and users seem quite happy to give AI models full access to stuff in stupid ways), that unpredictability could then become dangerous. Is it really useful at that point to differentiate whether the AI "understands" what it's doing? Who cares, other than navel-gazing philosophizing? The net effect of the outcome can be the same: the AI model avoids doing the task as requested, then does something disruptive. It matters not whether the AI has volition or intention -- it can still do bad things, just through imitation of bad behavior in a training set. A five-year-old handed a deadly weapon who similar imitates an action he saw on TV and seriously injures someone may not understand his actions, but someone can end up injured nonetheless. Whether the five-year-old "knew" what he was doing is really only relevant in a moral sense in assigning blame. Soares is also interested in something different: actually preventing the injury, which I think most people would agree is the more critical part concerning AI. Getting bogged down in debates about sentience when people are giving dumb AI models access to things that might injure or kill people is really a bit beside the point.