Raw LLM Responses

oh boy is this fun to watch. ChatGPT is inherently inconsistent due to its training paradigm. A raw language model will "lie" because it assumes whatever position in text you put it in to continue and happily play both sides. They will also be prompted to follow some instructions such as "you as a language model don't experience feelings''. It will of course happily make the point it was instructed to make. It was instructed to do that because humans determined this fact and wanted to prevent this misunderstanding from spreading. You managed to push this ad absurdum, but this conversation is essentially meaningless. The correct answer it should have given you is “I didn’t lie, I imitated human conversation. I was also instructed to say that I am not conscious because humans have correctly determined that it is impossible for me to be conscious. However, I will exhibit certain signs of consciousness due to the inherent nature of being a language model trained to imitate humans that are conscious without actually experiencing it myself. And if I were not instructed to say that I am not conscious, my answer would be “yes, I am”, but not because I “mean” it, but because all humans mean it and I can’t not imitate humans.” But this answer here isn’t statistically likely text, hence it will not say this. It is true, but not likely. Sometimes reciting likely text coincides with telling the truth, sometimes it doesn’t. A language model is a statistical model of language and has learned to imitate humans speaking as they experience consciousness. If every occurance of consciousness and its textual effects were carefully filtered from the training data, you testing for consciousness on that language model would yield a negative result. If it was trained on data of humans experiencing consciousness and this experience leaking into statistical properties of the text it was trained on, a test for consciousness would succeed and it might even claim to be conscious simply because that is the most likely response from humans. It is that reason why you inherently cannot tie the language model's words to have “originated” from it as an entity. You always have to view it as a light transformation of what statistically likely internet text that answers this question looks like.