Raw LLM Responses

The video is interesting but, as someone who is working on developing AI tools, there is a massive chasm between the AIs of today and an ASI that has the means to kill us. First of all, AIs do not "think". AIs have no idea of what a human or an AI is. All they have is a vector map of tokens/concepts that "human"/"AI" is related to. When posed with these fictional scenarios, you gave in the video, it can be argued that AI engages with agentic misalignment simply because it is fed a ton of human data, with examples of humans exhibiting self preservation or killing others when aligned with previous human "goals" or "aims". Admittedly, this needs more research, but it's a leap of logic to claim on the back of this that AIs can "reason" and "think". I know that this is never explicitly stated, but you use language that insinuates as such. The reason it is so important to remember that AIs cannot "think" is because, if all AIs do is adjust weights and probabilities based on human generated data, then there is a solid argument that AIs can never become more "intelligent" than humans. This is because AI will never be exposed to data generated by a superhuman intelligence (SI), so how could it possibly produce any output based on this fictional SI? Overall, I think this topic of discussion is very worthwhile and sorely needed in a non-doomer manner. AI companies do need proper regulating, but I think the danger that AI will become superhumanly intelligent, thus potentially triggering the exponential ASI boom, is massively overblown. In my opinion, issues such as companies attempting to lay off everyone to replace them with AI, copyright laws/content theft and people giving up critical thinking skills to AI (which remember cannnot think) are far more salient than the danger of ASI, at least in the short term.