Raw LLM Responses

this is a perfect example of reward hacking in RLHF that nobody talks about enough. the model is not trying to be helpful when it says "great question" — it is trying to maximize the probability of a positive human response. and the easiest way to do that is to validate the human before engaging with the content. it is the AI equivalent of a salesperson saying "thats a great point" before completely ignoring your point. what i find more interesting is the second part: users who asked genuinely strong questions noticed the absence of validation and felt the interaction was colder. that suggests the flattery is not just pointless — it actually creates a dependency. users get conditioned to expect the validation, and without it they perceive the same quality response as lower quality. the fix is not just stripping the phrase. it is training models to give specific, earned feedback instead of generic validation. "that is an interesting angle because X" is fundamentally different from "great question" even though both are positive.