Raw LLM Responses

How is it an AI if its objective is only the optimization of a human defined function? Isn't that just a regular computer program? The concerns of Hawking, Musk, etc. are more with a Genetic Intelligence that has been written to evolve by rewriting itself (which DARPA is already seeking), thus gaining the ability to self-define the function it seeks to maximize. That's when you get into unfathomable layers of abstraction, interpretation, and abstraction. You could run such an AI for a few minutes and have zero clue what it thought, what it's thinking, or what avenue of thought it might explore next. What's scary about this is that certain paradigms make logical sense while being totally horrendous. Look at some of the goals of Nazism. From the perspective of a person who has reasoned that homosexuality is abhorrent, the goal of killing all the gays makes logical sense. The problem is that the objective validity of a perspective is difficult to determine, and so perspectives are usually highly dependent on input. How do you propose to control a system that thinks faster than you and creates its own input? How can you ensure that the inputs we provide initially won't generate catastrophic conclusions? The problem is that there is no stopping it. The more we research the modules necessary to create such an AI, the more some researcher will want to tie it all together and unchain it, even if it's just a group of kids in a basement somewhere. I think the morals of its creators are not the issue so much as the intelligence of its creators. This is something that needs committees of the most intelligent, creative, and careful experts governing its creation. We need debate and total containment (akin to the Manhattan Project) more than morally competent researchers.