Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's talk about LLMs

b-list.org

Post reply on HN
Print view View original

81–90 of 201 posts

Re: Let's talk about LLMs

#81

Post by kasey_junk »

skydhash wrote: post #65
ekidd wrote: post #60

Earlier quoted context omitted.

The unfortunate reality is that a lot of software does have hard constraints. And a lot of these constraints are "gatekept" by regulators, compliance policies, insurance companies, etc. If someone slops together a medical record system, and leaks a bunch of PHI, there will be consequences, even in the US. Similarly, good luck getting insurance against cyber attacks without a SOC2 audit or equivalent. I've had this co…

That’s why the biggest proponent of LLM tooling are managers and entrepreneurs (aka people that are incentivized to reduce costs due to salary costs). But anyone that has to keep the system running and doesn’t want to wake up in the middle of the night is rightly cautious.

I’m literally tasked with reliability engineering and llms are far and away the biggest boost in that in my career.

Re: Let's talk about LLMs

#82

Post by guille_ »

cfloyd wrote: post #59
paganel wrote: post #50

Earlier quoted context omitted.

> , it tests N(x) faster. It does? You mean "it tests itself faster", which is not really a test now, is it?

I use one model for coding and another writing tests for that very reason. It’s surprisingly good at TDD

I find this fascinating because it's the sort of anthropomorphism that betrays a fundamental understanding of what an LLM is. Language models are not people. You can just achieve the same thing with a fresh context window. The only solid technical reason you'd want a different model is if you find a certain model produces better code and another produces better reviews. Nobody has really tested this, of course.

Re: Let's talk about LLMs

#83

Post by empthought »

strange_quark wrote: post #52
radarsat1 wrote: post #44

Earlier quoted context omitted.

> It's wildly better at tight and limited scope than large scale changes but even then I would rather code it myself. One thing I would like to see is the use of LLMs for smarter semi-manual editing. While programming I often need to make very similar changes in several places. If the instances are similar enough I can get away with recording a one-off keyboard macro to repeat, but if there are differences that are t…

I've had a similar thought. A super refactor feature would be amazing, but wouldn't fit into the current zeitgeist of agent everything. Hopefully as the hype starts to die down and prices go up, we'll get some of these smaller, more targeted features.

You don't need a special feature for this. Just tell the coding assistant what to do.

Re: Let's talk about LLMs

#84

Post by leptons »

kelnos wrote: post #27

>> Within just this group the ratios between best and worst performances averaged about 10:1 on productivity measurements and an amazing 5:1 on program speed and space measurements! > (although I’m personally skeptical of the “10x programmer” concept, the software industry overall does seem to accept it as true) To be fair, this statement from Brooks doesn't entirely match with the "10x programmer" we talk about. My…

There's no such thing as a "10x" programmer, and anyone who uses it doesn't know what they're talking about.

10x relative to what exactly? It's not a statement grounded in any kind of reality.

Re: Let's talk about LLMs

#85

Post by wavemode »

atleastoptimal wrote: post #73
slopinthebag wrote: post #32

Earlier quoted context omitted.

I didn't get the sense that the author is nervous. What I tend to see are people who are nervous that going all-in on LLM workflows might not have the payoff they are expecting, and are becoming increasingly fanatical as a result. Just one more harness bro. Just one more agentic swarm. Please bro, just one more Claude Max subscription. Please bro.

Complaining about every one off issue with LLM's ignores the bigger picture: they are getting better every month and there is no fundamental reason why they wouldn't surpass humans in coding. Everything else is secondary. All I would need from an LLM doubter is evidence that at tractable software engineering task LLM's are not improving. The strongest argument against the increasing general capabilities of LLM's are…

Your logic is flawed because, a thing can improve for an infinite amount of time while never surpassing a certain limit. It's called an asymptote.

That being said, I don't even think that arguing about this from a mathematical perspective is a worthwhile use of time. Calling something an asymptote in the first place requires defining a quantifiable "X" and "Y", which we don't even have. What we have are a bunch of synthetic benchmarks. Even ignoring the fact that the answers to the questions are known to regularly leak into the training data (in other words, it's possible for scores to increase while capabilities remain the same), there's also the fundamental fact that performance on benchmarks is not the same thing as performance in the real world. And being able to answer some arbitrary set of arbitrary questions on a benchmark which the previous model couldn't, does not have a quantifiable correlation to some specific amount of real-world improvement.

The OP article focuses on research papers which assess real-world impact of LLMs within software organizations, which I think are more representative.

I wouldn't call myself an "AI doubter" - I use LLMs every day. When you say "doubter" you're not referring to "AI" in general, or the fact that AI is helpful or boosts productivity (which I believe it does). You're rather referring to the very specific, very extraordinary claim, that LLMs will surpass humans in coding. If that's the case then yeah I'm a doubter, at least on any foreseeable timescale.

Re: Let's talk about LLMs

#86

Post by leptons »

aspenmartin wrote: post #41
slopinthebag wrote: post #32

Earlier quoted context omitted.

I didn't get the sense that the author is nervous. What I tend to see are people who are nervous that going all-in on LLM workflows might not have the payoff they are expecting, and are becoming increasingly fanatical as a result. Just one more harness bro. Just one more agentic swarm. Please bro, just one more Claude Max subscription. Please bro.

You say this as though performance has not followed a very clear and extremely rapid improvement in a startlingly short amount of time. You’re definitely right that people adopt agentic workflows and are disappointed or worse, but the point is the disappointment has already reduced substantially and will continue to do so. We know this because we know the scaling laws, and also because learning theory has been around…

You say this as though AI company debt has not followed a very clear and extremely rapid ballooning in a startlingly short amount of time.

It's the "YOLO" of business strategies.

Re: Let's talk about LLMs

#87

Post by hackable_sand »

leptons wrote: post #84
kelnos wrote: post #27

>> Within just this group the ratios between best and worst performances averaged about 10:1 on productivity measurements and an amazing 5:1 on program speed and space measurements! > (although I’m personally skeptical of the “10x programmer” concept, the software industry overall does seem to accept it as true) To be fair, this statement from Brooks doesn't entirely match with the "10x programmer" we talk about. My…

There's no such thing as a "10x" programmer, and anyone who uses it doesn't know what they're talking about. 10x relative to what exactly? It's not a statement grounded in any kind of reality.

It's a transparent exercise in ego-stroking to justify one's commitment to capital incentives.

Re: Let's talk about LLMs

#88

Post by threecheese »

kasey_junk wrote: post #80
yellowapple wrote: post #77

Earlier quoted context omitted.

Since when are SOC audits not a meaningful thing?

If soc audits are driving your development process you are doing it backwards. And _certainly_ a time is coming when just using the llm will be soc compliant.

I’d think any company big enough or working in certain markets which has a Compliance Officer cares about this; regulations are a legitimate business risk, and software integration contracts have security control compliance requirements which very much impact the sdlc.

Would you have the same reaction to requiring an approval for a production deployment? That’s driving the development process.

—-

Also jfc I need to cool it with the buzzwords, sorry I just got home from “talk like this all day” $job

Re: Let's talk about LLMs

#89

Post by threecheese »

kasey_junk wrote: post #81
skydhash wrote: post #65

Earlier quoted context omitted.

That’s why the biggest proponent of LLM tooling are managers and entrepreneurs (aka people that are incentivized to reduce costs due to salary costs). But anyone that has to keep the system running and doesn’t want to wake up in the middle of the night is rightly cautious.

I’m literally tasked with reliability engineering and llms are far and away the biggest boost in that in my career.

To be fair, that’s a role which most companies don’t have; even if they have a titled “SRE”, many times it’s a sysadmin in a hat, looking very tired and nervous. It must be fun right now tho

Re: Let's talk about LLMs

#90

Post by sov »

tptacek wrote: post #46
oytis wrote: post #34

Earlier quoted context omitted.

The article addresses exactly this objection. Most importantly, it quotes that AI coding tools have a detrimental effect on software stability - which is basically raison d'etre for our profession. When it produces more robust software and handles on-call shifts better than humans, I will consider programming done.

I'm excited to read the first cogent piece making this point that doesn't devolve to gatekeeping, a detached and vaguely hostile professional software developer telling people with a newfound capability to solve practical problems for themselves with new software that they don't or shouldn't want the thing that they want, because whatever it is they come up with won't be "fit for purpose" until blessed by the guild,…

I think you've misread the article, specifically the purpose of the Brooks quotes. They're clearly not to denigrate anyone for wanting a more useful or convenient way of generating software--they're specifically about the "LLMs will obliterate software engineering as a profession" claims put forth by many LLM marketers. In fact, Brooks is never once mentioned in the "Power to the people?" section of the essay.

Generally, the whole point of the "Power to the people?" (and to some extent the "On being left behind") section(s) is to underscore the two antithetical claims made by many LLM marketers: 1. LLMs are so powerful and so natural and easy that someone with no experience can create amazing software, and 2. LLM usage is a core skill, one that if you don't begin training now you'll be left behind.

Obviously, both of these can't be simultaneously 100% true--either it's easy enough for the non-programming layperson to successfully generate software for an intentional purpose, or, LLM assisted programming is a skill you need to train to avoid professional obsolescence in modern society. So, the article disagrees with the majority of both claims, and accepts a weakened/minor portion of each: 1. LLM output is easy to generate but accurate prompting matters, and 2. when used for software development professionally, some amount of skilled human intervention does indeed seem necessary. And now these two claims do align.

However, if professional software engineers who work with and read code constantly, armed with the best software practices to aid LLMs we can determine, cannot use modern AI tools without shooting their feet off at relatively frequent rates, certainly you'd expect the layperson who must put an even greater amount of undue faith in the validity of the results to be at extremely high-risk of foot-shooting. It's not "gatekeeping" to forewarn people against unwarranted trust in LLM output, nor is it "gatekeeping" to suggest that modern tech communicators/marketers describing an overly flowery LLM tooling landscape might be doing people a disservice.

Post reply on HN