Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The LLM Critics Are Right. I Use LLMs Anyway

theocharis.dev

Post reply on HN
Print view View original

221–230 of 330 posts

Re: The LLM Critics Are Right. I Use LLMs Anyway

#221

Post by annzabelle »

bigstrat2003 wrote: post #182
qsort wrote: post #62

Earlier quoted context omitted.

> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM I think there are two different claims here: - developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims. - LLMs give no productivity uplift at all, which is much…

I have seen no productivity uplift at all from LLMs. They are at least about neutral these days (they used to be a productivity drain ), but no gains at all. By the time I get done reviewing the code to make sure it hasn't done anything crazy, I've spent the same amount of time I would've taken to write the code myself. The only people I personally know who claim productivity gains are getting those gains by complete…

I find Claude is shockingly slow. If it were genuinely instantaneous, it would be worth it, but I often find myself twiddling my thumbs while waiting for it to cook.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#222

Post by annzabelle »

Diogenesian wrote: post #152
GenerocUsername wrote: post #140

Earlier quoted context omitted.

If cars did not exist, I would be healthier, able to walk and run many times further due to constant cardio exercise. I would still travel much less distance. And just like cars, LLMs will reshape the world to the point that our brains could not even get us to the supermarket because soon it will be 5 miles away and require a car ( or at least a local LLM bike )

FWIW a 5 mile walk is not a long walk. Humans are very much designed to make that trek without too much trouble. Lots of the world has people walking similar distances for fresh water, to go to school, etc. The point being that cars also reshape your perception of the world. METR had trouble replicating their dev productivity study because devs have already become much whinier about doing things "manually." Things th…

I had the chance recently to do several of New Zealand's Great Walks (3-4 day scenic wilderness treks with more expensive huts than the rest of the hut system), and I highly recommend doing something similar if you can swing it. Walking 15-30 km a day (9-18 miles) for several days while carrying supplies on your back really gives you new context for what you can do and what life was like without cars.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#223

Post by thin_carapace »

Aurornis wrote: post #203
breadzeppelin__ wrote: post #100

Earlier quoted context omitted.

I am both a career developer and experienced team manager. from first hand experience the 1.5x im getting from AI is not flowing down to my team / org because why would i output 50% more when the pay environment and leadership are already underwhelming. That additional 50% productivity goes completely to side projects built on my second computer between 9-5 tasks

> because why would i output 50% more when the pay environment and leadership are already underwhelming The productivity uplift measured in some of these reports is from more tasks being done, not from developers giving themselves more idle time during the day. If we do see job losses from AI, my feeling is that it’s going to be concentrated in people with an attitude like this one where workers try to remain anchore…

thats absolutely true regarding wysiwyg projects like a small website. aside from temporary cost savings, systematically ignoring institutional knowledge in favor of mass output is not gonna do humanity any favors ..

Re: The LLM Critics Are Right. I Use LLMs Anyway

#224

Post by singingtoday »

swiftcoder wrote: post #117

> I am a little bit scared to say this too: last month I spent almost 10k USD on tokens. It sounds so insane. What are people actually doing with all these tokens? I use LLMs pretty heavily for development, and I'm rarely spending all the tokens that come with a $10/month OpenCode Go subscription...

I have 3 max20 subscriptions and usually max at least two of them out.

I'm doing lots.

I'm building a social media site that nobody will ever use. I'm printing an ancient game to Linux and expanding the features (such as adding hw acceleration where it isn't needed). I'm troubleshooting crash logs for an Xbox game that has maybe 50 active players. Im building a distributed customer service system that provides LLMs, RAG, tool calls for lots of integrations (Gmail, calendar, sheets) (this one actually has several paying users). I'm building games, and releasing them on the Google marketplace (play store?)

But what used the most tokens was a kind of agent swarm I experimented with to implement a complex desktop application for very specific data manipulation. That.. half worked.. but it's buggy.

I made several internal tools at my company that are actually getting used by pretty much everybody. That's my most successful work I think. Oh, and I created a backup deployment in Azure for my companies product because .. we had the azure credits, so why not?

Re: The LLM Critics Are Right. I Use LLMs Anyway

#225

Post by anon7000 »

deadbabe wrote: post #145
qsort wrote: post #62

Earlier quoted context omitted.

> I will just point out the benefit is not as obvious as you think. Developers have consistently overestimated LLM I think there are two different claims here: - developers overestimate productivity gains, which is a solid finding in many of these studies. Skepticism of extremely large productivity gains is warranted and I flatly disbelieve "10x uplift" claims. - LLMs give no productivity uplift at all, which is much…

Developers are cashing in on the productivity gains. Meaning instead of using the increased productivity to do more work, they just become lazier or do fun irrelevant side projects instead, where as before there was just not much time for such things. I have definitely procrastinated on work simply because I know I can swoop in with an LLM and do it all in 15 minutes, whereas before I would have spent a few hours. Th…

Exactly, I finally migrated that old service to typescript and cleaned up some really annoying tooling, for example. I’m more productive and can get quick fixes out very quickly. We’re also seeing agents help identify incidents faster than human investigation can. But big structural things that are hard to integrate still take lots of time.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#226

Post by singingtoday »

adamas wrote: post #73
elicash wrote: post #71

Earlier quoted context omitted.

That's the cost of one employee (and not an expensive one). Do you think it's impossible he's getting more value from the AI than he would from a single employee?

If it's an employee, which I doubt but ok, it's an employee you constantly have to probe, talk to, give instructions to, so, not a really good employee.

Depends how you use the tokens. It's not 10k, but I have an agent do an investigation automatically when I get paged, so by the time I get to the computer I have a nice summary. But that's costing me like $5/mo at most.

But you could use tons of autonomous agents for all kinds of things. Probably shouldn't, but you can.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#227

Post by andyferris »

erwald wrote: post #29
mohamedkoubaa wrote: post #7

>as a senior, you don’t need juniors anymore. The mundane tasks, at least I find that a lot of people agree with that one, can be fully outsourced to an LLM Master craftsmen didn't take on apprentices to give them chores.

Master craftsmen paid apprentices almost next to nothing, and they were often contractually guaranteed to stick around for many years, so the teaching was a kind of wage and also a cost that could be recuperated later on. (The apprentice even often had to pay the craftsman to take them on.) None of those things are true for junior software engineers, who are paid to contribute and can leave at any moment. Also, yes a…

Not to mention, the physical trades are not stopping taking on apprentices because... LLMs can't build a house? Not even the mundane bits.

It will be interesting how to get the financial incentives to train juniors in knowledge work to line up with the fact that every new generation needs to be trained.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#228

Post by mkw5053 »

coldtea wrote: post #171
tablarasa wrote: post #137

Earlier quoted context omitted.

> How does the value of LLM-produced 100% branch coverage compare to no-LLM 100% branch coverage? This is such a salient question. Sometimes (definitely not always) the test suites produced by LLMs are so trivial it's scary. Coverage can be an illusion for sure.

I have been seeing such trivial shit getting submitted and deleting them from the test suites, but it's disheartening to see that the devs didn't have the better taste to see those LLM "tests" are so trivial they border on noop before submitting them...

We try to use types as much as possible. And if we can't, then lint rules, etc. Basically, get everything as far left in the SDLC as we can.

And, if/where we need tests, we write the source so they are few, high value, and complementary. Like actual unit tests, not complex with stuff like mocks just to generate trivial coverage.

Re: The LLM Critics Are Right. I Use LLMs Anyway

#229

Post by paulhebert »

cognitiveinline wrote: post #163
eamonnsullivan wrote: post #159

Earlier quoted context omitted.

Well, I doubt that. I mostly work from home. But also I live in a big city (London) with a working public transport system.

I dont know what to tell you. I guess - correct your misconceptions? Carbon footprint of commute is in KGs of CO2. A day's worth of AI, depending on model, is ~15-100g of CO2. Heck food you eat has more than a few Kgs of footprint everyday.

Does that include the training?

Re: The LLM Critics Are Right. I Use LLMs Anyway

#230

Post by Terr_ »

jmartrican wrote: post #59
xnorswap wrote: post #31

Earlier quoted context omitted.

I've been using regex decades, but it never really stuck to do anything too complex, it was the perfect intersection of difficult and infrequent. ( And also variable - PCRE vs others customisations / non-regular parts, etc ). I am very glad that I can now just ask claude for a regex to achieve my intent. Does it mean I'll never master regex? Yes it does, but decades has shown that was unlikely to ever happen anyway.

Regex came up so infrequent that I found myself referring to documentation whenever I needed to use it. But I always wondered, what are the jobs or roles that use it so often that they have mastered it.

That's somewhat alien to me, I suppose because over the years I just never stopped using regex in search/replace tools, which is daily exposure regardless of what code needs to be written.

Sometimes the patterns are dirt-simple, like `/\bfoo/` to find things that start with foo, or a `/foo.?bar/i` to try to find multiple variations of FooBar and fooBar and foo-bar and FOO_BAR. Other contexts outside an IDE include browser dev-tools, exploratory SQL queries on the database, searching centralized logs, and occasionally turning lines of text into spreadsheets via CSV.

None of that guarantees deep expertise of the weirder tricks, but it absolutely keeps the general knowledge "warm."

Post reply on HN