Earlier quoted context omitted.
> I'd say I'm getting 10x more non technical business advice in multiple areas than I'm getting with the technical stuff. Probably in large part because you can't verify when its wrong since that isn't your area of expertise, so those advice looks better to you. You should assume your org will be the same quality as a program written by a non-technical person using an LLM if you do it that way.
This is a critical insight. Whenever I ask an LLM about a topic I’m knowledgeable about, I can detect that it spews a load of nonsense. Talking to people in other fields, e.g., history and literature, they say it’s the same for them. LLMs are only useful if you know the topic you’re asking questions about. Otherwise, they generate so much nonsense that you cannot verify, that trusting it becomes a liability.
People are falling into the same trap of accepting authoritative sounding advice as true in an area they are not experts in.