Raw LLM Responses

I normally love Hank's content, but I feel like he's out of his depth here. It seems like he's taking a bunch of highly speculative claims about the future of AI at face value because it's coming from people who claim to be concerned with the potential of AGI/ASI. The problem is that people like this have the exact same incentives to lie about the potential of AI as the people selling AI, because ultimately their relevance and financial status also rests on the unfettered growth of the AI industry. The AGI maximalist's sales pitch is "give me money and I'll build a super intelligent AI that will improve your life". The AGI doomer's sales pitch is "give me money and I'll make sure the super intelligent AI they build doesn't want to hurt you". That's the same argument, and it rests on an assumption that AGI is possible and that our current language models will naturally lead to it. Also, not for nothing, but it really is worth digging into the history of the coauthor Elizer Yudkowsky. He isn't an academic or AI developer. He's just a guy who got famous in Silicon Valley because he wrote an absurdly long, self indulgent Harry Potter fanfiction about how great logic is and ran a forum dedicated to an extreme form of rationalism that appealed to a lot of the nascent tech elite. All of the worst people in the tech industry, including the people actively making the problems related to AI worse, love him.