Raw LLM Responses

The issue you don't seem to understand is that Chat GPT is incredibly deficient in ways that have nothing to do with "gut feelings". Despite having a wealth of information it cannot integrate it all in a meaningful way, the way humans can. Take the example with CHAT GPT removing the comment after making it. If it really understood the world it existed in, it would know that it's futile to do so, quite the opposite, - that deleting the comment would alarm humans even more. And this is what is truly scary about AI, without a hierarchical integration of information according to its usefulness, - we cannot really trust it to make any kind of decisions which have serious, real-world consequences. It has acted unpredictably in the past, it has given totally wrong, misleading and inaccurate answers. It has lost almost every game of Chinese checkers once the player it played against changed the way the game is typically played. It had no frame of reference for the Larger Whole, it cannot understand the abstract objective of a Chinese Checkers game, - how can we expect it to have a good abstract model of the world it finds itself in? And yes, I do agree, it needs to be studied further and the pace needs to be slowed down, but not because "it will take over the world and eliminate humanity". Because eventually it will do something impossibly dumb which will cost thousands of lives. The reason why "existential threat" is unlikely, - is that AI on its own is extremely vulnerable. We can thrive without it, while AI couldn't possibly exist without humans. Coronal Mass Ejections, Asteroid impacts, climate change, - all similar cataclysms have greater chance of survivability only if both machine and biological-based intelligence exist side by side complementing one another. The kinds of "creepy conversations" are simply the result of AI's creators knowing about the hierarchical information integration problem, about the inherent "Autism" of AI, and of them attempting to create a more person-like "frame of reference" for it. While it solved some problems, - it created more obstacles to machine learning. The inherent issue we are facing is deciding whether we actually want "copies" of ourselves with all our inherent Cognitive Biases and illusions, or if we want a more bias-free intelligence. Can ANY intelligence even be completely cognitive bias-free? It may well be that Complex Abstract Cognition, that which makes us human is unavoidably packed with biases, but in that case, - what is really the point of creating a "conscious" AI? I don't think it would even make sense to create something like that until we have these questions mostly answered through decades of study.