Raw LLM Responses

I can confidently say that LLM can't replace people using an anecdotal experience. I joined a big tech last year that was in the middle of a huge layoff. I was handed over a project that sounds like something the AI would orgasm over: a lot of slides, initial design docs, system design docs, team docs, evaluation results and a "working" code base - working in the sense that if you feed it something, it spits out something that looks correct. There was a major bug caused from data query that crumbles the whole thing. The previous user used AI. When I fed the relevant docs to Claude 4.6, it happily verifies that the code works. Only after extensive debugging did I find the bug, and only then Claude accepted the error. Admittedly this is caused by data quality issues, but I suspect that this is and will be very common, especially with legacy systems.