Software engineer here trying to start a startup. The amount of business advice GPT-4 has given me is immeasurable. I'd say I'm getting 10x more non technical business advice in multiple areas than I'm getting with the technical stuff. Kind of like an AI CEO advising me on what to do next.
Probably in large part because you can't verify when its wrong since that isn't your area of expertise, so those advice looks better to you.
You should assume your org will be the same quality as a program written by a non-technical person using an LLM if you do it that way.