Live data from Hacker News

I think you might be fooling yourself with AI

louwrentius.com

61–70 of 174 posts

Re: I think you might be fooling yourself with AI

#61
Quotes study from early 2025. Same group redid study now shows opposite result. “Fooling yourself” indeed.

Anybody actually using AI in anger (to get paid) will tell you AI was pretty bad early 2025. It wasn’t usable till June. By December it was writing most of my code - with significant reviews by me. Now it’s basically getting it right. YMMV.

Re: I think you might be fooling yourself with AI

#62
post #9

These articles are so off from reality to the point that it can't be taken seriously at all. I keep wondering how they keep popping at HN. At least the author admit he is biased..

I think there's a simple dichotomy based on use cases. For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear. But for things you aren't good at, they're zomg amazing. So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey what's all the hype about?' But if you're doing things outside your domain then it's a revolutionary game-changer.

This also explains why an independent dev can proclaim a 10x productivity boost or whatever, and actually seem to be showing that - while major companies dumping obscene amounts of $$$ on tokens don't seem to be have much to show for it.

Re: I think you might be fooling yourself with AI

#63
We've been debating how to measure software engineering productivity since long before LLMs. It depends on so many different factors. Scrum is sort of the only tool we have that comes close, but we all hate it. Aside from that, all we really have are personal anecdotes.

Here are my personal anecdotes for the week. I occasionally ask AI to write some SystemVerilog test bench code. This is an area where I have deep and long experience. AI has never once saved me time with that. Just yesterday, however, I was writing a tricky Makefile, something I rarely do, and AI helped a lot.

Aside from productivity, this article mentions ethical and financial issues with AI. That doesn't stick with the subject that the title of the essay implies, but it's a more concrete thing to discuss and I wish we'd spend more time on that. I think that's where people really are fooling themselves.

Re: I think you might be fooling yourself with AI

#64
This is where I find it lacking:

1. I am not yet convinced that I am more productive with AI for complex tasks. But, I am fairly certain for trivial tasks, or bureaucratic tasks AI has lessened my cognitive load (data point of one).

2. Does AI run on money, or does it run on energy? Money is the means by which we facilitate exchange. But in reality, the energy in the universe is ginormous. The fundamental thing to keep AI going is available in excess, all other things are negotiation.

3. Is AI bad for the environment or is it the way we obtain energy bad for the environment? The trend seems to be that energy is getting cheaper and cleaner. The world is not static, but will move to accommodate a world with AI.

Re: I think you might be fooling yourself with AI

#65
The responses to this post made me really sad.

Sure - it was light on the meat, But it's an opinion blog post, not a full article. the author ventures to cite their refs, acknowledges potential bias.. way more than you get with a tonne of front page articles (many of which probably should have been short opinion pieces like this)

Sure - it's not particularly notable or high entropy enough to save, but it's far from offensive or bothersome ...yet all I'm seeing (at time of posting) is defensive dismissals, ad hominems, and echo chamber pablum

Re: I think you might be fooling yourself with AI

#66
From TFA>>I'm open to suggestions how this is going to play out. Close to a trillion dollars invested in AI. Where will the money come from to recuperate all these investments?

If investors are looking for a 50% return, they need the investment to be worth about $1.5 trillion. If they're looking for a 100% return, it needs to be worth about $2 trillion. That's still an enormous hurdle, but it's much smaller than suggesting AI needs to somehow generate trillions of dollars in cash simply to recoup the original investment.

Re: I think you might be fooling yourself with AI

#67
post #43

I read this headline and thought "METR study"... and sure enough, a few paragraphs in: > Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower. I think you might be fooling yourself if you build your entire worldview concerning the productivity benefits of AI-assisted programming around that one study from one orga…

What study do you base your worldview on?

Four years of personal exploration of the technology.

(I had an argument on Lobste.rs the other day about why only believing aspects of software engineering that have survived peer review doesn't make sense to me: https://lobste.rs/s/5kgenk/reviewing_ai_code_is_not_viable_a... )

Re: I think you might be fooling yourself with AI

#68

Earlier quoted context omitted.

> AI costs aren’t going to stay this high. They will go down. People said this a year ago, then all the companies admitted they were incinerating cash and had to increase to token pricing. Consumers basically mutinied and managed to push back a bit, but that is the struggle right now - to keep prices from increasing 25+ times, not to see them decrease. If you're premising this on some future technology like ASICs, ev…

People said this a year ago A year ago Opus 4.1 was $75 per million tokens, and now Opus 4.8 is $25 and Fable is $50.

[deleted]

Re: I think you might be fooling yourself with AI

#69
post #9

These articles are so off from reality to the point that it can't be taken seriously at all. I keep wondering how they keep popping at HN. At least the author admit he is biased..

"These articles are so off from reality to the point that it can't be taken seriously at all" So you disagree with the quote from Feynman? (the quote that forms the basis of the article) I'm genuinely curious to know what part of the article you feel is "so far off from reality"?

There is no longer, in my opinion, any scope for reasonable debate that AI models like Fable 5 are net productivity increases for many, if not most, if not ALL, programming tasks. Anyone can, or should be able to, prove this to themselves in about 5 minutes.

And so if someone like the author is trying to argue the contrary, that leads me to believe they either: (1) have not bothered to spend even 5 minutes validating their belief or (2) do not care to test their beliefs against reality, for whatever reason. Either one is a bit of a red flag.

Re: I think you might be fooling yourself with AI

#70
post #34

> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower. METR reran the study early this year and, while they caveat it, this time they found a speedup, which is consistent with subjective estimates of productivity also having increased -- the simplest explanation is that subjective estimates exaggerate, but there'…

"Unfortunately, given participant feedback and surveys, we believe that the data from our new experiment gives us an unreliable signal of the current productivity effect of AI tools."

That's more than a caveat. That says, this study is unreliable. And why? "The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI."

So, maybe people are fooling themselves, just like this article suggests (I almost wrote asserts, but saying, "I think..." isn't really an assertion)

Post reply on HN