Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eight Myths on Software Engineering and GenAI

queue.acm.org

Post reply on HN
Print view View original

211–220 of 269 posts

Re: Eight Myths on Software Engineering and GenAI

#211

Post by TeMPOraL »

a_bonobo wrote: post #16

>On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them; This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around th…

> And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does.

Sure it can, turn up the "temperature" a bit.

There's this notion that human "jumping" is magic. It's not. It's all based on inputs. Including unrelated inputs, past inputs, and feeding yourself your own thoughts.

The hard part is not the ability to make conceptual jumps. That's just random search. The hard part is discrimination: whether a given mental jump is "creative" or "insane". Iterated, the problem is that of balancing between the two failure modes: relax your thinking too much, and you'll start thinking nonsense thoughts; tighten it too much, and you'll be just following immediate-term rewards and obvious thought trains. It takes time to find that balance, and plenty of people at various points err in one or the other direction (e.g. small kids in particular tend to err on the "crazy non-sequitur side", but that's because they're learning the basics of reality and social interactions).

Re: Eight Myths on Software Engineering and GenAI

#212

Post by TeMPOraL »

pdhborges wrote: post #192
dmurray wrote: post #176

Earlier quoted context omitted.

It's a very silly point to make to AI researchers specifically. If they don't work on those projects, the AI won't advance and won't magically be able to replicate the work in "one to three years".

Can you imagine scenarios that would make it less silly? I will give an example: - The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?

That company or their staff is probably focusing its funds on getting itself acquired/acquihired by the top dog.

Re: Eight Myths on Software Engineering and GenAI

#213

Post by camdenreslink »

baobeta2907 wrote: post #75

This is actually true at my company. They expect employees to be 10× more productive now that we have AI.

You can be 10x more productive if you don’t verify anything the LLM generates and just YOLO it into your codebase. I wouldn’t recommend it.

Re: Eight Myths on Software Engineering and GenAI

#214

Post by TeMPOraL »

iammrpayments wrote: post #168
guybedo wrote: post #25

Earlier quoted context omitted.

although, if i'm out of tokens and have to wait a full day, i won't bother doing some things manually because the day i'll spend doing something won't take more than 1 hour the next day when tokens are available again.

According to the article that part being automated isn’t more than 18% of your day anyway

At this point at least half of your day is spent doing work that you shouldn't be doing in the first place, but over past decades companies saw it fit to eliminate specialized roles with legible paychecks, and smear the work they did on everyone else until it disappears from the books.

Self-service and office suite software is largely responsible for this.

Re: Eight Myths on Software Engineering and GenAI

#215

Post by jreynar »

simonw wrote: post #4

> We already know developers don’t actually spend most of their time writing code, with studies at Microsoft and elsewhere showing it’s closer to 14 percent. Anyone else finding they're spending more time writing code (or at least driving agents to write code) now? 14% used to feel about right for me - I'd spend the rest of the time researching approaches and libraries, planning things out in issues, or sometimes jus…

[flagged]

Re: Eight Myths on Software Engineering and GenAI

#216

Post by discreteevent »

TeMPOraL wrote: post #211
a_bonobo wrote: post #16

>On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them; This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around th…

> And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does. Sure it can, turn up the "temperature" a bit. There's this notion that human "jumping" is magic. It's not. It's all based on inputs. Including unr…

> The hard part is discrimination: whether a given mental jump is "creative" or "insane"

That's what they mean by LLM's can't jump. They mean it can't make a creative jump. Their example is Einstein's Theory of Relativity - It's not a random jump.

Re: Eight Myths on Software Engineering and GenAI

#217

Post by vladimirralev »

TeMPOraL wrote: post #211
a_bonobo wrote: post #16

>On my visits to the Bay Area, I would ask AI researchers or interns why they are doing their current research or projects, when in a year or three agentic LLMs could probably do them; This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around th…

> And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does. Sure it can, turn up the "temperature" a bit. There's this notion that human "jumping" is magic. It's not. It's all based on inputs. Including unr…

100% agree. Plus you can easily instruct it to use inspiration from unspecified unrelated and counterintuitive concepts at random and in parallel. The "jump" is not "on" by default because it would waste tokens on high risk paths, not because it's incapable. It's very capable if you are willing to wast some tokens.

Re: Eight Myths on Software Engineering and GenAI

#218

Post by discreteevent »

simonw wrote: post #141
t-writescode wrote: post #111

Earlier quoted context omitted.

> Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly. That is *exactly* the sort of area I *wouldn’t* blindly trust AI, there’s a huge security boundary there. What if the AI is doing string concatenation with user-provided data???

Once you've seen the AI not make mistakes like that a few dozen times you start to trust it not to mess that up in the future.

Is this really a rational strategy for something whose nature is to be right most of the time and then spectacularly wrong a much lesser amount of the time?

Re: Eight Myths on Software Engineering and GenAI

#219

Post by discreteevent »

simonw wrote: post #114
unknownfuture wrote: post #104

Earlier quoted context omitted.

The article mentions that coding is only a fraction of dev time and thus accelerating that part of the job can only create incremental gains, and oh PS, lines of code and similar metrics are a bad way if measuring dev productivity, anyway, and we've known that for decades. The OP claims AI accelerates non-coding parts of the job, too, and so the article is misguided. I ask for evidence. In response you give me... cod…

I don't have anything else to hand that I can think of. I don't keep a time tracking diary.

> I don't keep a time tracking diary.

That's why Microsoft did a study (referenced in the article), where they measured the time spent on things, so they could get to the truth of how much time people spend coding using "a shape of evidence would you find convincing"

Re: Eight Myths on Software Engineering and GenAI

#220

Post by germandiago »

Yopolo wrote: post #202
germandiago wrote: post #191

Earlier quoted context omitted.

> I would say yes and it will scale So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? Tasks it has not been tweaked for. When I deviated from the average, and not really weird things, when programming, the output was way worse than average stuff. And this is an explicit target of AIs nowadays. I think you are missing a lot of details here, honestly. > Exac…

> So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? I'm pretty sure we will solve this issue. Either already through World Models or another architecture. It could also be, that we just need a 10 or 100 Trillion Parameter model to match so many generic ways of solving tasks and keeping the concept in the LLMs 'head' to solve it that it will just emerge wit…

> I'm pretty sure we will solve this issue. Either already through World Models or another architecture.

Please elaborate. How? With which technique? Currently the only path forward is to feed more data and tweak for specific situations (fitting, basically). How does that help in the general case or in new situations with current tecchnology (LLMs, concretely). Noatter how far you get, this is not a general or reliable solution. It van only simulate more generality or more reliability by training and tweaking. Nothing else. At least, with this paradigm.

This does not mean they will not be useful. What I challenge here is the AGI or singularity. We are far from that.

> I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.

I have been a lead and an architect also for years at different position. I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. And if it is, then you have to dumo so much context that it is better to go do it yourself. There is a cost to that also actually. It is not just so "dry and technical" the knowledge. Maybe yes to learn Java patterns or C++ constructors or the like.

But not for "given this situation with all these specifics", which solution would you bet on? Probably the LLM will give you a shitty REST API that is not what u need at all.So u tell the AI. It gives u something else generati g 30-50% of "decorated code". Now it seems to workso you use it. Now you do this every day. Come back in 2 months. You generated a lot of fat.

Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.

Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.

TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns. But I saw some and use a prompt with limited access and the best I can take out for my speed + control when coding is tech discussions to decide on it, error catching, test generation, one-off scripts... But never "make an app like this or that". If I ever do that (I did it a couple of times) is for scaffolding and later throw away 70%.

Namely, to see something that runs on screen quickly. But later you need to spend time yourself as usual. Not a bad thing, just that this is not what you deliver and need the work done. Iterations etc.

Post reply on HN