> How far does the rabbit hole go for verifying what is going on?

That is up to the person claiming the output. The point is not what the person knows about the output. It is the fact that laying claim to that output, means they are responsible.

To dramatise the point, they don't get to put computer viruses out on the internet and then later say "Oh, the LLM must've done it..." and walk away.

Use the tools you use. But the code is yours.