Raw LLM Responses

I teach legal research to foreign students who already have law degrees in their home countries and come to the US to get a US law degree. The ones in my class want to take the New York bar exam. Remember, these are students who have been practicing lawyers and sometimes judges in their home countries. I had one ask during orientation this year if they could just upload all the reading to ChatGPT to summarize it. Many of them were using AI on really simple legal research exercises that were ungraded, as if I didn't: a) know what the answer to the exercise was; b) have accounts with all of the major LLMs and was able to cut and paste the exercise into each one to see what kind of slop each one put out. In one case, a student was offended that I would think she used AI, until I hovered the mouse over the link she provided that ended in "source=ChatGPT." Some of them just copied the AI answer word-for-word. Some made an effort to tailor their answer, but never bothered to look at the source material, so a whole bunch of them made the same mistake on a regulations exercise because the model they used pulled from a blog post from 2011 and used the link to the relevant regulation included in the blog post, which was also from 2011, and thus had the completely wrong citation because that entire subsection of the CFR had been re-numbered since then. I'd even offered them amnesty early in the semester to re-submit answers without the use of AI, since it's against our policies. But now I have to report a bunch of them to the dean. As foreign students, they're paying full price for tuition. I wish I had that kind of money to throw away. Also: good fucking luck on the bar.