Raw LLM Responses

​@mez8384 The main property of "in-context learning" which allows large enough language models to model abstractions around predicting the next token/word in a sentence, was NOT intentional. Apart from the calculus / backpropagation necessary for artificial neural network training: The intentional parts are the transformer model design ("prior" which looks across a context window of words/tokens in text to find base meaning in it + from this: abstractions / metaphors / concept patterns over later parts of the model: or layers), its initial pre-training process and then instruction training (finetuning). Can optionally add "think for longer" / reinforcement learning step here. Alongside much improved computer hardware + huge amount of data on the internet, since the 70s/80s. And btw, natural selection *did* allow for this to happen whether implicitly (or explicitly, *very* unlikely though) - as evolution is the actual process behind it; which has reached where we are today. A abstracted process which can freely adjust physical processes and incentives among them at a great enough skill and speed, can act similarly to this - this is a probably more grounded analogy to the original metaphor. I would argue that humans + our approach to economics are doing this on planet Earth, right now. So the divergence exists and can continue to exist, even in other things which have language and intelligence equal to or greater than humans do (the equal point is not necessary, just a baseline). The problem is that we are still terrible at intentionally placing discrete rules and goals (specified in language basically) into such transformer model-based AI systems as mentioned. Language model prompting has so many edge cases, and relies on interpretation through the "predict the next token" task... And these rules would not be guaranteed to always remain over time as of the current training process, or be modifiable in terms of consistent language or discrete concepts, as we don't even know if there is such a consistent analogy or structure in the parameters of the model (a related concept is called 'equivariance'). With current models, we might only be able to get somewhat close to this in terms of statistics / probability of occurrence. But that may not be good enough for our safety over time.