Raw LLM Responses

My problem with the "philosophers will figure out whether this qualifies as 'reasoning'" line is that it still assumes way too much about what the models are actually doing, that "oh, yeah, maybe it *technically* might not be reasoning but hey, it's definitely in that direction". No, it's also possible that LLM chain of thought-style reasoning is ultimately a dead end, that it can sort of do things that look like reasoning but too much is fundamentally missing to lead to even human-level intelligence. Clever Hans wasn't a sign that you could one day teach horses to do math, he was responding to subconscious physical cues unrelated to any numbers, and it's entirely possible you can mimic basic reasoning with syntactical analysis but not anything more advanced, or being far too inefficient to practically do so, like building with bricks and no mortar. And for the obvious counterpoint, yes, technology improves and things get better, but not always and not in every category; fusion and superconductors with reasonable requirements have been just twenty years away for decades, but it always turned out the challenges were much harder than expected. That's why I take so much issue with the above, language affects how we think about things and evaluate evidence, and handwaving "reasoning" as ultimately a philosophical point avoids confronting whether the thing is what we think it might be, or whether it's a very clever facsimile that can't succeed with larger tasks. Talking like LLMs actually understand anything, even with all the caveats in the world, predisposes us to evaluate it in those terms. (I have a whole bunch of rants on these topics and the misrepresentations of what AI is actually doing these days, the aside about hidden states or "knowing it's being tested" being two others, but I have limited time and energy to put together a YouTube comment :P )