Raw LLM Responses

I saw earlier that people were struggling to have ChatGPT write scripts for telephone calls because it couldn't comprehend that both parties weren't aware of each others surrounding environment, and so when something happens to one of the individuals, chatGPT assumes the other is aware of it. This would suggest that it has no (or little) theory of mind. Then again, a response from another individual showed a Fraser script where another character enters the room mid-conversation and is not aware of the context of the conversation, whereas the other two are. This seems like a demonstration of theory of mind, though it may be such a common plot device that ChatGPT could be argued to be simply mimicking ToM much like a parrot doesn't understand the words it vocalises. It should be fairly simple to test theory of mind though. There are many measures that are typically administered to children which might be useful if you wanted to give it a go. I had a quick scan of [this](https://www.frontiersin.org/articles/10.3389/fpsyg.2019.02905/full) synthesis on ToM measures, which states there are some "read aloud" measures, and some which should be easy to adapt to text form. You can view the supplemental material for a list of the measures they found in their search.