Raw LLM Responses

Interestingly from a comsci pov, the LLM being able to verbatim spit out paragraphs tells me that on the technical side, they overfitted their model and it just memorised and never made connections to abstract ideas of the articles when training. OpenAI's "prompt hacking" defense is a bit of a PR spin. While it's true the NYT had to use specific prompts to "induce" the recall, the fact that the data is stored in a way that can be recalled verbatim is, by definition, a failure of the model to fully abstract the information. Which means the model isn't "piecing bits and pieces here and there" based on the prompt, but just figured out that "this prompt" should output this exact paragraph for maximum reward. That tells me also that they didn't just go to NYT website and parse the article once, but they went through many many MANY different sources, like web crawlers and archive sites and shared articles on social media, and did not filter out duplicates. So the model was repeatedly exposed to the same article over and over and started memorising It also means that openai did not have access to much training data, but threw in a lot of money on hardware to make the LLM huge. So instead of breaking the input and memorising abstract ideas, it had too many layers and started memorising instead (like a student with photographic memory won't study and understand the concepts of "multiplication" but just memorise pages and pages of multiplication table)