Raw LLM Responses

You're kinda right, but explainability in ML is important and especially in applications such as software used in criminal investigations as described in the video. Imagine getting arrested and tried over a crime you didn't commit, and when your defense cross examines the prosecutor over their argument, they shrug their shoulders and says "idk man, our new software we're trialing says it's him. I'm not a data scientist, neural networks work in mysterious ways! But this stuff is great conviction rates are up 99% so it must be good" And then you subpoena an Israeli ai firm asking for source code and weights for an expert witness who does understand to analyze, but they tell you to get stuffed because it's proprietary data and they donated to the judges reelection campaign and you didn't so you have to take the plea deal.