Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eight Myths on Software Engineering and GenAI

queue.acm.org

Post reply on HN
Print view View original

171–180 of 269 posts

Re: Eight Myths on Software Engineering and GenAI

#171

Post by geraneum »

simonw wrote: post #105
geraneum wrote: post #90

Earlier quoted context omitted.

Does the code get reviewed? How do you deal with increased amount of code that may need to be looked at?

I review the code that matters - anything security adjacent or that's an API that will be used by other code in the future. I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works. Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks O…

> I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.

Good example of what not to review if you're working on your hobbies. Also exploratory can sometimes be done this way. However, this ultimately boils down to how you approach programming as an engineering discipline, including your responsibility for the outcome.

> I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.

This doesn't generalize well. If you drink raw milk, or if you don't wear your seatbelt, or if you don't escape your user input correctly, you'll probably be fine, but I really hope aspiring programmers/engineers don't take this attitude towards any serious task. One should always examine their biases, tools' failure modes, etc. regardless of how many times something didn't fail.

> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.

One [should] review the code they're responsible for. In a team, people usually assign you (or ask you) to review code, and the work is divided accordingly. If the code isn’t reviewed by the code owners, it’s a problem, not something inspiring!

> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.

Have you considered that the sheer amount of code being generated is what makes thorough review infeasible, not that it’s a desirable approach?

Re: Eight Myths on Software Engineering and GenAI

#172

Post by Yopolo »

germandiago wrote: post #161
Yopolo wrote: post #152

Earlier quoted context omitted.

No we don't know your 'points' The Hallucinations are becoming less, significantly by now. It also might be already were it is cheaper for one of the big few companies to spend millions and billions to teach the LLM / creating the training data necessary for an LLM to do something which it is not yet good enough due to the fact, that they sell this capability then to everyone who wants to use this capabilitiy. We hav…

> The Hallucinations are becoming less, significantly by now. Yes? What is the mega-solid technique that is used for it? Armies of people using curated data and reviewing it by hand? That is exactly one of my points: shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that? I am not denying there is some use (a lot of uses!) for thi…

> shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?

I would say yes and it will scale. It will either happen through central LLM just paying for it and scaling it up to everyone on the planet (literaly) OR by the agentic layer every big business is building into their systems.

You needed some human to use your tool optimized for their company? With agentic layer you no longer need this. And if you look at companies like Google, they were pushing this notion for ages already because they saw an adoption problem of more 'complex' tools and trying to make it simpler and easier. Now you can act from the other side too.

> We do not have even enough compute power for current technology.

Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.

> The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time.

It doesn't need to be perfect, it only needs to be better than the avg human. And the current LLMs are already better than aat least 1-2 people in my team.

> Now add the human factor of requirements and back and forth

Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.

And all of this can be copy and pasted. There is only one harness needed which becomes the expert security reviewer and tomorrow everyone can have it.

I'm still discusing progress with LLMs with people and still not everyone is using it or playing around with harnesses or developgn an agentic layer. We still have a lot of work to do to even see how good it will become while it already is really good.

People are already borred of AI today and making wrong decisions based on the current level of AI while i think we will see continues progress for years.

Re: Eight Myths on Software Engineering and GenAI

#173

Post by griffiths »

lelanthran wrote: post #153
griffiths wrote: post #87

Earlier quoted context omitted.

In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI.

> In my experience, not everyone can really do this stuff by typing. I think you need to be creative, resourceful, inventive, open minded and have ideas how to approach the typing/prompting. I see many people struggle in using AI. The problem, for the profession, is that the set of people who can really do this stuff by typing is close to "all of them". I'm not seeing anyone struggle with using AI. I see struggles fr…

I can attest to the other side as well, that I have seen professional software developers outputting code of lower quality than AI. And I would say that during my career (18 or so years) I have met a small number of quality software developers or engineers. Although that might be because I was not in Silicon Valley where most of the smart/hotshot engineers converge.

Re: Eight Myths on Software Engineering and GenAI

#174

Post by griffiths »

t-writescode wrote: post #111
simonw wrote: post #105

Earlier quoted context omitted.

I review the code that matters - anything security adjacent or that's an API that will be used by other code in the future. I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works. Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks O…

> Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly. That is *exactly* the sort of area I *wouldn’t* blindly trust AI, there’s a huge security boundary there. What if the AI is doing string concatenation with user-provided data???

I agree with a commenter above/below (depending where this comment lands). For some time models won't do this. And any review from review agents would caught this. For most of the AI programming there needs to be a more stricter (automated) review process now. Most SAST tools would caught this type of security issue.

Re: Eight Myths on Software Engineering and GenAI

#175

Post by mfru »

I've found that using LLMs for significant amounts of code generation completely drain the result from any dopamine I would get doing it myself.

Have others noticed this as well? This is going so far as to me losing interest in side projects because I have "lost touch" with the code base.

Re: Eight Myths on Software Engineering and GenAI

#176

Post by dmurray »

germandiago wrote: post #127
a_bonobo wrote: post #16

>On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them; This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around th…

> This is such a weird point to make I think it is a great point to make, because if everyone really believed that AIs will do everything without human intervention in a handful of years, as the marketing repeats again and again (AGI, singularity, etc.) and have been saying for years... why then get bothered? Because we DO know LLMs have their hallucinations, limitations, perform tasks not previously seen way worse t…

It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".

Re: Eight Myths on Software Engineering and GenAI

#177

Post by t-writescode »

mekael wrote: post #122
simonw wrote: post #105

Earlier quoted context omitted.

I review the code that matters - anything security adjacent or that's an API that will be used by other code in the future. I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works. Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks O…

I've found that engineers on a large team do read every line, mainly due to the fact that the skill levels run the gamut from intern to lead, and only 1 or 2 people out of 12 might have knowledge of the application being modified. It's actually getting worse due to "AI code bloat", for example I have 16k lines of code to review across 3 apps by the end of this week. Normally it would be a quarter of that, but what Cl…

Goodness, how is that being tolerated? I guess it can’t be stopped without a lot of political capital; but 16k lines of code is HUGE, and I cannot imagine that it’s actually 16k lines of value - I’ve written whole new subsystems of a product in fewer lines. Are these all written in an exceptionally verbose language like Go or Java? Are they VERY well documented? Are they doing things they shouldn’t be doing???

Re: Eight Myths on Software Engineering and GenAI

#178

Post by qsera »

simonw wrote: post #103
the_af wrote: post #77

Earlier quoted context omitted.

Isn't this falling into the "lines of code" trap TFA mentions? More code written is not a good measure of productivity. It could be garbage, or redundant code, or simply not addressing the real or more pressing needs, it could be building the wrong thing, etc. Like TFA mentions, it's been known for decades LoC is a misleading metric for productivity. It's one of the lessons of software engineering.

I'm planning an article at the moment in defense of lines of code. Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument. In my specific case, lines of code for my published open source projects is a metric that I trust, because I have high standards for those. I have plenty of other projects where I'll accept poor quality, unreviewed code (almost all of http…

> Saying "lines of code are a bad measurement" is too easy! It's about time someone presented an opposing argument.

Why though? The only entities who would be interested in such an implication is the AI marketing.

No one real cares about more or less lines of code. But everyone cares about decreasing the complexity of the implementation. Sometimes that means more LOC and sometimes it means less.

Re: Eight Myths on Software Engineering and GenAI

#179

Post by lentil_soup »

>a “good” workday, engineers spent 18 percent of their time “coding” (not including bug fixing, testing, etc.)

that's such a weird metric, why exclude bug fixing and testing? depending on the phase of the project I might spend 100% of my coding time bug fixing

Re: Eight Myths on Software Engineering and GenAI

#180

Post by jurgenburgen »

blackqueeriroh wrote: post #117
AdieuToLogic wrote: post #82

Earlier quoted context omitted.

>> How are you going to prompt the LLM or validate its output if you don't understand the requirements? > Your PM can write the ticket, and your QA can test the output. If your PM can write the ticket and your QA can test the output, why not cut out the middleman by having the PM write the coding agent prompt(s)? And if your PM can write the coding agent prompt(s), why not cut out the QA group by having the PM write…

Because customers have better things to do.

Do they though? I assume they’re also getting replaced by LLM in this hypothetical.
Post reply on HN