Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev

I had a similar question that I posed to an ally in senior leadership. The answer, obviously I guess, was metrics. Assuming your teams didn’t just FAFO through it, you could reconstruct what the expected outcomes were vs actuals. And follow that with sound recommendations about where it could be used along with some thoughts about the future state (when the tools are faster or cheaper or whatever).

Another commenter had a great note, that you should take this opportunity to advance your career, given you are in an excellent position to do so. It made me giggle, but then think that you really need to make sure your PoC was executed thoughtfully and seriously, that you understand what the SoTA is wrt using assistants and their various modalities (chat, agent, copilot etc) and despite your expertise it was a no go. Because if you don’t, somebody else who is taking it dead seriously is going to take that commenters advice and demonstrate the value that you didn’t, and this may reflect poorly on you. Execs are getting hype from their back channels or vendors like I have never seen in my career, and you are going to go against that. ($bigco perspective there)

"It hasn't for the same reason it hasn't made you a 10x exec".

Because writing prompts is hard.

Most AI coding tools can generate a Todo app from a small prompt. This is because that problem is well understood.

When you try to use AI coding tools on your own projects you need to start writing a prompt that teaches the AI about your current architecture and decisions.

So the initial prompt is large.

Often the task needs knowledge of other files in your project. You can add them by hand or some AI tools will search the code base.

The prompt is now huge.

When you run that prompt you may or may not get what you expected.

So now the issue is how much time do you spend getting the prompt correct vs just writing the code yourself.

This area is brand new and there are very few resources on how to use AI coding tools effectively.

I have yet to see one demonstration of effective AI coding tool use on a project of reasonable complexity.

When you say ChatGPT, do you mean 4o, o1, o1 pro, or…?