Raw LLM Responses

Reasoning is a huge breakthrough that is less than a year old. There is also no evidence that scaling doesn't work well anymore, the "wall" is currently an economic one. Agents are breakthroughs in productivity, not foundation model performance. Ultimately, productivity is what drives growth in the space beyond hype, so this is still a good thing. And people have been trying new things. There are tons of invisible advances, if you think today's models are gpt2 with more parameters and training data you're just wrong. Even in the way you think of breakthroughs, there have been many proposals about fundamentally improving the basic transformer like sparse attention or Titans/ATLAS.