I would look at why you're outsourcing this thinking to HN instead of putting together data and an understanding of your own environment. Or just ask the LLM's to write it for you.
Free answer from HN: Don't believe the hype.
Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
11–20 of 28 posts
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#12Biggest cause is ineffective reasoning in a large context in my experience (more than 350 lines). Im my experience, ChatGPT breaks down when it needs to consider more than 350 lines and its performance is sloppy before that. To get solid performance out of it, I essentially need to specify the important areas and changes as well as the desired approach. That being said, I’ve found it has cut my development time for m…
a few examples where AI has been extremely helpful for work related things:
- writing complex sql queries for throw-away code, i.e. need to create a few charts from big query - writing internal front end tooling - polishing any internal messages - helping design docs with clarity - helping unblock highly technical areas where i haven't had any experience - helping learn the sdk from new vendors - helping read api documentation, i.e. does the instagram api allow for ABC
areas where it's been mildly helpful:
- helping to write verbose methods that i'm too lazy to write. e.g. maybe a custom date parser that takes a datetime and outputs some custom text according to a spec
areas where it hasn't been that helpful
- building features e2e. as you mentioned, the context window issue is really the crux here. - building simple code like a crud api. our system, like all others, has its own idioms and spinning up a new crud api is easy enough. getting an llm to write it is do-able but by the time i've edited things to make it consistent with the rest of our system, i've lost the benefit
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#13Why ask HN? Have you actually tried using the tech? Then you should have the answers. There are lots of different types of software development and in some areas gen AI will be extremely useful, but a net negative in others. Only you can answer the questions about your environment.
> Why ask HN?
bc hn is a community of smart ppl. I'm sure these discussions are happening at most software companies so I was just curious how others are dealing with it
> Have you actually tried using the tech? Then you should have the answers. There are lots of different types of software development and in some areas gen AI will be extremely useful, but a net negative in others. Only you can answer the questions about your environment.
yes, daily. :) here's my rough answer to my own question:
- as another commenter mentioned, context windows aren't large enough to capture every detail in our system - coding is actually the easy part of the job. Generally speaking, once we know the details of the task (i.e. building table X with N fields, build a service that does XYZ, etc), the coding portion doesn't take a lot of time. So the cost saving of outsourcing that to Claude/GPT/etc can be helpful, but it's not a game changer - team alignment is hard and figuring out what to build is hard - we lose eng time on the unknowns that gen ai doesn't really help with: debugging an issue with a vendor's sdk, identifying the cause of a race condition, figuring out how to mitigate a bot issue, etc.
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#14Why is the onus on you, why don't the execs explain why it _would_ 10x the efficiency of their feature factory, if they're so smart?
The obvious answer is that generating code is not the hard part of building products.
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#15Why is the onus on you, why don't the execs explain why it _would_ 10x the efficiency of their feature factory, if they're so smart? The obvious answer is that generating code is not the hard part of building products.
This is a classic example of org-politic power play, and a blame game. Programmers are not skilled in this kind of internal corporate warfare.
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#16AI is like hiring jr developers, have they any evidence this has worked prior? Why would they expect an equivalent to work?
How did they come to this belief in the 10x AI developer? Get them to question their base assumptions, ask them to justify their expectations
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#17you would be wise to in fact do the exact opposite. claim it does more than 10x. ask to spearhead the AI transformation of your company. promise huge cost savings. make your bag. and who knows, maybe you'll actually deliver on those promises as a side effect. and what if not? remember, the senior eng team is there as a convenient scapegoat ...
@iExploder has been a consultant in a previous life - this is a more realistic plan than it sounds
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#18Just be clear about the current technology in language a 5 year old can follow. Use short sentences and connect the dots.
1. AI is business jargon for Large Language Models.
2. LLMs are predictive models using plain text.
3. LLMs are not more accurate or creative than what they are fed.
4. LLMs can provide excellent documentation for prior encountered problems but cannot provide original solutions to new problems.
5. A more effective means at cost reduction is to reduce/eliminate regression. This provides the same benefit as LLMs but without sacrificing creativity.
6. This is an opportunity for executives to reduce expenses and dramatically and simultaneously radically increase product quality. Lead the developers to increase test automation coverage and increase execution speed.
7. The alternative for the developers is their replacement by LLMs. LLMs cannot replace people but cost so much less they make up the difference if product quality remains marginal or maintenance cost remains high.
That is how you do it. Small sentences, use numbers to explain the finances, and play devils advocate.
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#19I would look at why you're outsourcing this thinking to HN instead of putting together data and an understanding of your own environment. Or just ask the LLM's to write it for you.
I tried Copilot and found the answer it gave was unsatisfying, the best para I can lift is "While gen AI can assist with generating code snippets, documentation, and automating certain tasks, it may not be as effective in solving complex, domain-specific problems that require deep understanding and critical thinking." all accurate but not really compelling to somebody who is under the spell of magic beans.
This famous essay
https://worrydream.com/refs/Brooks_1986_-_No_Silver_Bullet.p...
points out why 10x is a pipe dream with any technology. The core of the argument is that software development involves a number of steps, let's say
* requirements gathering
* systems support (build/dependencies/version control)
* software architecture
* user interface design
* database design
* coding
* testing
* documentation
* user training
* deployment
[1] the argument wouldn't be too different if one of these was 30% of the work, and one certainly isn't
Re: Ask HN: How to explain to execs why gen AI hasn't 10x'd feature dev
#20Biggest cause is ineffective reasoning in a large context in my experience (more than 350 lines). Im my experience, ChatGPT breaks down when it needs to consider more than 350 lines and its performance is sloppy before that. To get solid performance out of it, I essentially need to specify the important areas and changes as well as the desired approach. That being said, I’ve found it has cut my development time for m…
a few examples where AI has been extremely helpful for work related things: - writing complex sql queries for throw-away code, i.e. need to create a few charts from big query - writing internal front end tooling - polishing any internal messages - helping design docs with clarity - helping unblock highly technical areas where i haven't had any experience - helping learn the sdk from new vendors - helping read api doc…
It might be time to retire those idioms? A realistic outcome of AI-driven development is code is no longer a long term asset. Patterns like DRY are beneficial because humans have cognitive limits, and many humans over long timeframes need to maintain complex software systems. Maybe patterns like locality of reference are better than DRY for short context window AIs, and maybe we need to start looking at how we would manage many smaller bits of code, with lots of duplication, just like we attend to developer experience today with code analysis and declarative build and deploy pipelines.
If you HAD to deal with 1,000 individual functions - let’s say due to hardware and organizational limitations - how would you manage the obvious risks? Would that be net-net cheaper than a fleet of microservices and development teams?