Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use AI code tools as collaborators, not crutches

newsletter.oberai.dev

Post reply on HN
Print view View original

61–70 of 78 posts

Re: Use AI code tools as collaborators, not crutches

#61

Post by quantadev »

prmph wrote: post #58
quantadev wrote: post #54

Earlier quoted context omitted.

I'll spend like 10 minutes crafting a prompt that explains a new feature to be added to my app. I explain it in enough detail, with zero ambiguity, such that any human [senior] developer could do it. Often the result is 100s of lines of code generated, and well over 95% of the time the code "Claude 4" generates is exactly what I wanted. I'm using VSCode Github Copilot in "Agent Mode", btw. It's able to navigate aroun…

Your response does not address the example I gave. Sure, if what you are doing is a variation on something that's been done to death, then an LLM is faster at cutting and gluing boilerplate together across multiple files. Anything beyond that and LLMs require a lot of hand holding, and frequently regress to boot

I can't tell you how many times I've seen people write shoddy ambiguous prompts and then blame the LLM for not being able to read their minds.

If you write a prompt with perfect specificity as to what you want done, an agent like "Github Copilot+Claude" can work at about the same level as a senior dev. I do it all day long. It writes complex SQL, complex algorithms, etc.

Saying it only does boilerplate well reminds me of my mother who was brainwashed by a PBS TV show into thinking LLMs can only finish sentences they've seen before and cannot reason thru things.

Re: Use AI code tools as collaborators, not crutches

#62

Post by prmph »

quantadev wrote: post #61
prmph wrote: post #58

Earlier quoted context omitted.

Your response does not address the example I gave. Sure, if what you are doing is a variation on something that's been done to death, then an LLM is faster at cutting and gluing boilerplate together across multiple files. Anything beyond that and LLMs require a lot of hand holding, and frequently regress to boot

I can't tell you how many times I've seen people write shoddy ambiguous prompts and then blame the LLM for not being able to read their minds. If you write a prompt with perfect specificity as to what you want done, an agent like "Github Copilot+Claude" can work at about the same level as a senior dev. I do it all day long. It writes complex SQL, complex algorithms, etc. Saying it only does boilerplate well reminds m…

You're still talking past my points. Look at the example I gave. Does it seem like the problem was due to an ambiguous prompt?

Even if my prompt was ambiguous, the LLM has no excuse producing code that does not type-check, or crashes in an obvious way when run. The ambiguity should affect what the code tries to do, not it's basic quality.

And your use of totalizing adjectives like "zero ambiguity" and "perfect specificity" tells me your arguments are somewhat suspect. There's nothing like "zero" and "perfect" as far as architecturing and implementing code goes.

Re: Use AI code tools as collaborators, not crutches

#63

Post by quantadev »

prmph wrote: post #59
quantadev wrote: post #56

Earlier quoted context omitted.

It's all about the quality of your prompts (i.e. your skill at writing clear unambiguous instructions with correct terminologies). An experienced developer can generate tons of great code 30x faster with an Agent, with each function/module still being written using the least amount of code possible. But you're right, the measure of good code isn't 'N', it's '1/N' (inverse), where N is number of lines of code to do so…

> Claude [almost] never breaks any of these rules. Well it does for me, frequently. An example is here: https://news.ycombinator.com/item?id=44126962

Not sure how Claude frequently fails for you, but everybody I know says it rarely fails. I'm definitely not claiming it's perfect tho.

Re: Use AI code tools as collaborators, not crutches

#64

Post by quantadev »

prmph wrote: post #62
quantadev wrote: post #61

Earlier quoted context omitted.

I can't tell you how many times I've seen people write shoddy ambiguous prompts and then blame the LLM for not being able to read their minds. If you write a prompt with perfect specificity as to what you want done, an agent like "Github Copilot+Claude" can work at about the same level as a senior dev. I do it all day long. It writes complex SQL, complex algorithms, etc. Saying it only does boilerplate well reminds m…

You're still talking past my points. Look at the example I gave. Does it seem like the problem was due to an ambiguous prompt? Even if my prompt was ambiguous, the LLM has no excuse producing code that does not type-check, or crashes in an obvious way when run. The ambiguity should affect what the code tries to do, not it's basic quality. And your use of totalizing adjectives like "zero ambiguity" and "perfect specif…

When it comes to zero ambiguity and perfect specificity here's how I define it: If I gave the same exact prompt wording to a human would there be any questions they'd need to ask me before starting the work? If they need to ask a clarifying question before starting then I wasn't clear, otherwise I was clear. If you want to balk at phrases like "perfectly clear" you're just nit picking at semantics.

Re: Use AI code tools as collaborators, not crutches

#65

Post by quantadev »

wizzwizz4 wrote: post #5
quantadev wrote: post #3

Experienced senior developers can spot and fix the slop instantly, while still getting a 30x productivity gain, while entry level or junior devs basically only have one "Filter" by which they determine code quality which is: "Does the code seem to work?". Unfortunately "Slop" will appear to work enough of the time to fool a Junior. Also the reason Junior devs get "slop" is because their prompts are "slop". They don't…

Whenever I can't just sit down and bash out code, it's because the design is wrong. These models are bad at design. I don't see where your 30×–50× could possibly come from.

That 30x math simply comes from spending 5min typing a prompt, and getting code generated that would take a human 2.5hrs to write. This means in the future most of a developer's time will be spent reviewing code, rather than typing it. Because AI will also be able to write the test cases too, so that effort [mostly] vanishes as well.

Re: Use AI code tools as collaborators, not crutches

#66

Post by tcoff91 »

quantadev wrote: post #57
fkyoureadthedoc wrote: post #19

Earlier quoted context omitted.

Bad/dumb developers don't get much of a boost in my experience working with a plethora of shitty contractors. Good developers aren't getting a 30x boost I don't think, but they are getting more out of the tooling than bad developers. The bottleneck is still finding good developers, even with the current generation of AI tooling in play.

It was when I started using Github Copilot in "Agent Mode" that my LLM productivity gains went from like 5x to 30x. People who are just using a chatbot get like 5x gains. People who use "Agent Mode" to write up a description of a new feature that would take several days by a human, but get it done in one click by an Agent, are getting 30x or more. The amount of pushback I got on this thread tells me most devs simply…

I’ve tried using agents. LLMs just can’t reliably accomplish the tasks that I have to do. They just get shit wrong and hallucinate a ton. If I don’t break the task down into tiny chunks then they go off the rails.

Re: Use AI code tools as collaborators, not crutches

#67

Post by tcoff91 »

quantadev wrote: post #56
prmph wrote: post #50

Earlier quoted context omitted.

It's ridiculous to equate lines of code to amount of engineering work or value. A massive amount of valuable work can result in a few lines of code. Conversely a millions lines of code can be useless or even have negative value.

It's all about the quality of your prompts (i.e. your skill at writing clear unambiguous instructions with correct terminologies). An experienced developer can generate tons of great code 30x faster with an Agent, with each function/module still being written using the least amount of code possible. But you're right, the measure of good code isn't 'N', it's '1/N' (inverse), where N is number of lines of code to do so…

What tech stack are you using? It matters a lot what tech you are using when it comes to how effective the LLMs are.

Re: Use AI code tools as collaborators, not crutches

#68

Post by wizzwizz4 »

quantadev wrote: post #65
wizzwizz4 wrote: post #5

Earlier quoted context omitted.

Whenever I can't just sit down and bash out code, it's because the design is wrong. These models are bad at design. I don't see where your 30×–50× could possibly come from.

That 30x math simply comes from spending 5min typing a prompt, and getting code generated that would take a human 2.5hrs to write. This means in the future most of a developer's time will be spent reviewing code, rather than typing it. Because AI will also be able to write the test cases too, so that effort [mostly] vanishes as well.

Unless your job is producing disposable software (e.g. single-use mobile games for short marketing campaigns), this comment suggests you don't know how to do your job. If a piece of the program takes 5 minutes to describe, but 2½ hours to write, you're spending your time in the wrong place, producing code that's legacy almost on day 1. Quoth https://quoteinvestigator.com/2014/03/29/sharp-axe/:

> The text presents to the wood cutter the alternative either to spend time in sharpening his axe, or expend his strength in using a dull one. Which shall he do? Wisdom is profitable to direct.

Sure, you don't need to sharpen your axe. Given a powerful internal combustion engine, you could drive a tank through the forest and fell many trees in rapid succession. But this strategy doesn't leave you with quality lumber, and leaves a huge mess for whoever comes after you (which may be your future self), and one day there won't be any trees left.

If your job is producing disposable software, be aware that you're using unpaid labour to do so. Some of the programmers who produced that AI's training data are struggling to eat and keep a roof over their heads. Act accordingly.

Re: Use AI code tools as collaborators, not crutches

#69

Post by quantadev »

tcoff91 wrote: post #66
quantadev wrote: post #57

Earlier quoted context omitted.

It was when I started using Github Copilot in "Agent Mode" that my LLM productivity gains went from like 5x to 30x. People who are just using a chatbot get like 5x gains. People who use "Agent Mode" to write up a description of a new feature that would take several days by a human, but get it done in one click by an Agent, are getting 30x or more. The amount of pushback I got on this thread tells me most devs simply…

I’ve tried using agents. LLMs just can’t reliably accomplish the tasks that I have to do. They just get shit wrong and hallucinate a ton. If I don’t break the task down into tiny chunks then they go off the rails.

This can definitely happen, because the context windows even in a great Agent can become flooded. I often do prompts like "Add a row of buttons at the top right named 'copy', 'cut', and 'paste'", and let the Agent do that, before I implement each button, for example.

The rule of thumb I've learned is to give an Agent the smallest possible task at a time, so there's zero ambiguity in the prompt, and context window is kept small.

Re: Use AI code tools as collaborators, not crutches

#70

Post by quantadev »

wizzwizz4 wrote: post #68
quantadev wrote: post #65

Earlier quoted context omitted.

That 30x math simply comes from spending 5min typing a prompt, and getting code generated that would take a human 2.5hrs to write. This means in the future most of a developer's time will be spent reviewing code, rather than typing it. Because AI will also be able to write the test cases too, so that effort [mostly] vanishes as well.

Unless your job is producing disposable software (e.g. single-use mobile games for short marketing campaigns), this comment suggests you don't know how to do your job. If a piece of the program takes 5 minutes to describe, but 2½ hours to write, you're spending your time in the wrong place, producing code that's legacy almost on day 1. Quoth https://quoteinvestigator.com/2014/03/29/sharp-axe/ : > The text presents to…

The 5min example is like a maximum/extreme case, yes. My average time spent writing each prompt is probably 30 seconds or less, and coding time saved per prompt like 25 to 60 minutes.

When I do spend minutes (not seconds) writing prompts, it's because I'm actually typing a "Context File" which describes with full clarity certain aspects of my architecture that are relevant to an Agent task set. This context file might have constraints and rules I want the Agent to follow; so I type it once and reference it from like 10 to 20 prompts perhaps. I also keep the prompt files as an archive for the future, so I can always go back and see what my original thoughts were. Also the context files help me do system documentation later.

Post reply on HN