Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts on LLMs and software development

martinfowler.com

Post reply on HN
Print view View original

11–20 of 422 posts

Re: Some thoughts on LLMs and software development

#11

Post by gjsman-1000 »

the_af wrote: post #8
gjsman-1000 wrote: post #4

For my money, I used this analogy at work: Before AI, we were trying to save money, but through a different technique: Prompting (overseas) humans. After over a decade of trying that, we learned that had... flaws. So round 2: Prompting (smart) robots. The job losses? This is just Offshoring 2.0; complete with everyone getting to re-learn the lessons of Offshoring 1.0.

> Prompting (overseas) humans [...] After over a decade of trying that, we learned that had... flaws. I think this is a US-centric point of view, and seems (though I hope it's not!) slightly condescending to those of us not in the US. Software engineering is more than what happens to US-based businesses and their leadership commanding hundreds or thousands of overseas humans. Offshoring in software is certainly a US…

The problem isn't that there aren't high quality offshore developers - far from it. Or even high quality AI models.

The problems are inherent with outsourcing to a 3rd party and having little oversight. Oversight is, in both cases, way harder than it appears.

Re: Some thoughts on LLMs and software development

#13

Post by Marazan »

krainboltgreene wrote: post #3

> All major technological advances have come with economic bubbles, from canals and railroads to the internet. Is this actually correct? I don't see any evidence for a "airflight bubble" or a "car bubble" or a "loom bubble" at the technologies' invention. Also the "canal bubble" wasn't about the technology, it was about the speculation on a series of big canals but we had been making canals for a long time. More impo…

The AI bubble also isn't about the technology.

Re: Some thoughts on LLMs and software development

#14

Post by atleastoptimal »

There are only 3 things that I have strong empirical evidence for with respect to LLMs

1. Routinely some task or domain of work that some expert claims that LLM’s will able to do, LLM’s start being able to reliably perform that task within 6 months to a year, if they haven’t already

2. Whenever AI gets better, people move the goalposts regarding what “intelligence” counts as

3. Still, LLM’s reveal that there is an element to intelligence that is not orthogonal to the ability to do well on tests or benchmarks

Re: Some thoughts on LLMs and software development

#16

Post by bko »

> Certainly if we ever ask a hallucination engine for a numeric answer, we should ask it at least three times, so we get some sense of the variation.

This works on people as well!

Cops do this when interrogating. You tell the same story three times, sometimes backwards. It's hard to keep track of everything if you're lying or you don't recall clearly so you can get a sense of confidence. Also works on interviews, ask them to explain a subject in three different ways to see if they truly understand.

Re: Some thoughts on LLMs and software development

#17

Post by catigula »

>I’m often asked, “what is the future of programming?” Should people consider entering software development now? Will LLMs eliminate the need for junior engineers? Should senior engineers get out of the profession before it’s too late? My answer to all these questions is “I haven’t the foggiest”

I just want to point out that this answer implicitly means that, at the very least, the profession is at least questionably uncertain which isn't a good sign for people with a long future orientation such as students.

Re: Some thoughts on LLMs and software development

#18

Post by tptacek »

sebnukem2 wrote: post #6

> hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. Nice.

In that framing, you can look at an agent as simply a filter on those hallucinations.

Re: Some thoughts on LLMs and software development

#19

Post by tptacek »

krainboltgreene wrote: post #3

> All major technological advances have come with economic bubbles, from canals and railroads to the internet. Is this actually correct? I don't see any evidence for a "airflight bubble" or a "car bubble" or a "loom bubble" at the technologies' invention. Also the "canal bubble" wasn't about the technology, it was about the speculation on a series of big canals but we had been making canals for a long time. More impo…

I don't know enough about the early history of the airline industry but there was very definitely a long series of huge bubbles in the railroad industry.

Re: Some thoughts on LLMs and software development

#20

Post by koolba »

rancar2 wrote: post #12

My favorite quote to borrow: “Furthermore I think anyone who says they know what this future will be is talking from an inappropriate orifice.”

Reminds me of the classic Yogi Berra, “It's tough to make predictions, especially about the future”.
Post reply on HN