Raw LLM Responses

I think most of what you're saying here is roughly true... I'm definitely not an AI doomer but I'm also not one to think that AI is going to change the world to the same degree that people financially invested in it are hoping. I use AI everyday for work, it is helpful, but even something like software engineering where it's almost perfectly structured to be useful. It still is not really that great. Even the code from the best models is mediocre. And limitations around context and the inability of the AI to actually learn as it goes outside of just expanding context, our real limitations to its ability to be useful. I think that the fundamental technology is not the correct technology.... In other words, I think this is a good stepping stone, but I think that whatever the technology will be that brings out more flexible, intelligent AI, it's probably not going to be transformers... At this point to me that feels quite clear The self-grounding stuff like deep seek, and the deep think type models where we're going to let it think without text being displayed, all of those things are going to make it better. And already have. But fundamentally if I'm working with a model, as soon as I clear its context because I start running out of tokens, it has to start relearning everything that it already learned about what I'm doing. It's also slow, and extremely expensive to run. I think that we've basically taken an imperfect solution of transformer models and basically push them nearly to their limits. Whatever brings us to better free-working AGI type models is absolutely not going to be transformers in my opinion. It's going to be a novel technology or a radical transformation of the existing technology. And it's definitely not going to be in a couple years.