The sources are reliable. There are prescribed sources for lumber products, fasteners, etc. in residential construction, at least in the US, that are just as accessible to you and me online as they are to structural engineers. Those same sources are what the engineers themselves rely on to do their work, or, more likely, most engineers rely on software that has those sources built in and don't ever reference the primary sources. All the information you need to make concrete, empirical decisions about things like posts in residential construction are available online and don't require an engineering degree to figure out. LLM's are great at taking the uncertain language you input and finding all the sources, and the calculations, for you so you don't have to spend hours digging around on Google to find a "document" you didn't know how to search for, that then has 600 numbers on it that you have to spend more time discerning which number is the right one to use. Or which calculation out of the infinite number out there is the right one for your case. Kind of like a skeleton key or maybe a dictionary that equips you with the language you don't yet know to get to the bottom of something you don't yet fully understand.
Btw, I would not trust an LLM to tell me how to build a suspension bridge. First, I'm unfamiliar with that space. Second, even if I was familiar, the stakes are, as you say, so high that it would be insane to trust something so complex without expert sign off. The post I'm specifically talking about? Near-zero stakes and near-zero risk.
I beg folks to always try and pierce the veil of complexity. Some things are complex and require very specialized training and guardrails. But other complexity is fabricated. There are entrenched interests who want you to feel like you can't do certain things. They're not malicious, but they sometimes exist to make or protect money. There are entire industries propped up by trade groups that are there to make it seem like some things are too complex to be done by laypeople, who have lobbied legislators for regulations that keep folks like you from tackling them. And if your knee-jerk reply is that I'm some kind of conspiracy theorist or anarchist all I'm saying is it's a spectrum. Suspension bridge with traffic driving over it --> should double, triple, quadruple check with professional(s); a post in a house supporting the entire house's load (exaggeration for effect) --> get a single professional to sign off; a post in a house that's supporting a single floor joist with minimal live and dead load (my case!) --> use an LLM to help you DIY the "engineering" to get to good enough (good enough = high margin for error); replace a light switch --> DIY YouTube video.
I am the king of long-winded HN posts. Obviously the time I took to write this (look, ma, no LLM!) is asymmetric with what you wrote, but I'm genuinely wondering if any of this makes you think differently. If not, that's cool of course (and great for the engineers and permit issuers!).