Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eight Myths on Software Engineering and GenAI

queue.acm.org

Post reply on HN
Print view View original

231–240 of 269 posts

Re: Eight Myths on Software Engineering and GenAI

#231

Post by _aavaa_ »

pdhborges wrote: post #192
dmurray wrote: post #176

Earlier quoted context omitted.

It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".

Can you imagine scenarios that would make it less silly? I will give an example: - The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?

Ask the researchers working on Deepseek. They seem to be doing pretty well for themselves.

Just because a AI will be able to do it in the future does not mean that us plebs will be allowed to have access to it. That alone is enough of a reason for smaller labs to keep going; having a seat at the table.

Re: Eight Myths on Software Engineering and GenAI

#232

Post by bob001 »

vladms wrote: post #156
lz400 wrote: post #97

Earlier quoted context omitted.

Coding was expensive in the sense that once you decided what to do, it took a few engineers months / years to do moderately complex projects. That's not true anymore. Therefore the risk of "coding the wrong thing" is less.

Lots os engineering projects took as long as you mentioned because they were horribly specified. Then the engineer either waits for an answer (which they generally do not get) or takes a decision (which might not be ideal without domain knowledge). Then you release and the customer suddenly starts explaining what they want. You can improve some parts with LLM (make more prototypes, iterate faster), but if humans are…

Specification is hard because you are trying to predict a future state in a vacuum. You need to do that because creating the product is very expensive coding wise. If instead you could instantly build to your current specification then you could iterate on the specification against a real product. Even if all that code is thrown out the end result would be a better specification.

Re: Eight Myths on Software Engineering and GenAI

#233

Post by bob001 »

gmueckl wrote: post #118
lz400 wrote: post #97

Earlier quoted context omitted.

Coding was expensive in the sense that once you decided what to do, it took a few engineers months / years to do moderately complex projects. That's not true anymore. Therefore the risk of "coding the wrong thing" is less.

Is it? The temptation to start without a thorough design is now much stronger because the implementation osnperceived to be cheap and easy to replace. But if you start building the wrong thing fast, you still get the right thing later than when you had checked properly at the start.

CICD and iterative development and agile work because interactive building leads to better product. Otherwise we'd all build perfect software using waterfall methodologies and release only once a year. If implementation is cheap then you implement, review and then re-implement. Not even iterate in the extreme case. Throw it away if needed and start from scratch if needed.

Re: Eight Myths on Software Engineering and GenAI

#234

Post by discreteevent »

simonw wrote: post #226
discreteevent wrote: post #218

Earlier quoted context omitted.

Is this really a rational strategy for something whose nature is to be right most of the time and then spectacularly wrong a much lesser amount of the time?

Yes, because the mistakes in code are easy to spot. I wouldn't use this to write me an unreviewed legal brief.

But you said that in this case you don't review the code because you trust the AI having seen it write that kind of code before.

Re: Eight Myths on Software Engineering and GenAI

#235

Post by lilbigdoot »

unknownfuture wrote: post #9
kylecazar wrote: post #7

I don't understand Myth 1 (Developers Spend Most of Their Time Writing Code). They quote a study in which developers report to spend 11-14% of their day coding. The rest is stuff like solution design and meetings. The insinuation is that AI can at most automate 14% of your day. The problem with this argument is that once you have code, some (not all) of the precursors to code go away.

Okay. Show me the evidence that AI has an impact on productivity when doing design work. Or reducing meeting load. My own experience is that AI doesn't tighten the design cycle, and in fact might extend it by encouraging gold plating.

Anecdotally I find them helpful for design and I don't find them nearly as helpful for coding (strange, given others experiences reported online)

I am working on a compiler for an OCaml inspired language (I am on a sabbatical) and used them a lot in the early phases to check my ideas and search for prior work. I use them less now (mostly debugging at this point), but have made use of LLMs to write some pieces like source map generation, a couple of small and well known algorithms that were new to me, and an analyzer to dump a schema from a typescript module. Other times I've tried to use it for other things, I have ended up regretting it because of subtle bugs

I don't think it made the design phase any shorter but it definitely helped save me energy in researching and made that period feel less like a fever dream. Design phases can be draining for me so that was welcome

Re: Eight Myths on Software Engineering and GenAI

#237

Post by Yopolo »

discreteevent wrote: post #223
Yopolo wrote: post #157

Earlier quoted context omitted.

LLM don't work on 'compressed data'. LLM compress data into their latent space which allows them to become general. They learn the concept of things and how to do them because this is better compression than learning concepts one by one. Which means, if an LLM 'learns' the concept of a poem, it can put everything into the formad of a poem instead of learning a billion poems.

> They learn the concept of things and how to do them because this is better compression than learning concepts one by one. When anthropic looked at how an LLM does addition it found it had some mental math heuristics that might or might not always work. The LLM hadn't learned the concept of addition. It had learned some heuristics that might work for some numbers. The result is that LLM's cannot add numbers reliably…

It learned a concept of a heuristic which made it smart enough for the learning reward.

Might be an architecture issue or a parameter size issue that it didn't learn to do math like a caculator.

But look at your own math skills: How many numbers / how big of numbers can you keep in your head? How far is this heuristic away from how much a human learned until you start using pen and paper or a caculator?

Re: Eight Myths on Software Engineering and GenAI

#238

Post by emp17344 »

TeMPOraL wrote: post #229
discreteevent wrote: post #216

Earlier quoted context omitted.

> The hard part is discrimination: whether a given mental jump is "creative" or "insane" That's what they mean by LLM's can't jump. They mean it can't make a creative jump. Their example is Einstein's Theory of Relativity - It's not a random jump.

Einstein didn't magically one day woke up and jump on relativity theory. He had prerequisites in terms of recent mathematical advancements (notation) and physical discoveries, and a job exposing him to a lot of lateral thinking, and time to bounce ideas around in his head. We don't know how many fruitless jumps he made before making one that we remember him for.

No one’s saying that relativity appeared out of thin air, but Einstein still made a conceptual leap that LLMs are incapable of replicating.

Re: Eight Myths on Software Engineering and GenAI

#239

Post by twister2920 »

simonw wrote: post #226
discreteevent wrote: post #218

Earlier quoted context omitted.

Is this really a rational strategy for something whose nature is to be right most of the time and then spectacularly wrong a much lesser amount of the time?

Yes, because the mistakes in code are easy to spot. I wouldn't use this to write me an unreviewed legal brief.

you literally just said you don't review this code!

Re: Eight Myths on Software Engineering and GenAI

#240

Post by twister2920 »

simonw wrote: post #227
champagnepapi wrote: post #207

Earlier quoted context omitted.

lol I think you’re setting yourself up for failure. Why? Just because something works doesn’t necessarily know you the boundaries of it. What scale does it work for? Will it crumble under load in prod? It works but allows cross tenant access (security issue) because security checks weren’t in the location you thought… Dangerous!

I've been waiting with baited breath for this to bite me for more than six months now. So far unbitten.

meaningless without telling us what you're working on
Post reply on HN