Raw LLM Responses

For this kind of task you would use an LLM as a classifier rather than as a generator. To explain the difference, say you had to determine if an online review was positive or negative. You can give an LLM two prompts, "Here is a negative review of our product:" and "Here is a positive review of our product:" Then give it the review in question. As the LLM parses each token in the text it will constantly try to generate the next token from there giving a probability distribution of what it thinks the next token should be. Somewhere in that list will be the actual next token and its associated probability for each of the prompts. By comparing the two you are basically asking for each word, "is this more likely to be something someone would say in a positive review or in a negative review." and using Bayes theorem you can determine which parts of the review are negative, which parts are positive, and which are neutral. This is pretty much the simplest version of an LLM as a classifier. An actual implementation of this idea applied to these emails would of course be more sophisticated. It would likely use a fine tuned model and to get an overall understanding of the response it would look at the model's embedding as it read. There would likely be no text prompt at all. To define those terms fine tuning just means training a more general model on data similar to the specific application it will be used in, and an embedding is the list of parameter weights that represent the state of the model. If you say "Tell me a story" to chatGPT, then after it finishes parsing that text it will have applied weights to each of the billions of parameters within its model such that it embeds the idea of just having been asked to tell you a story. There is a different embedding for reciting the communist manifesto and a different embedding for being asked what crayon is tastiest after a flirtatious conversation about polymer chemistry in which it's pretending to be a purpl