Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I think you might be fooling yourself with AI

louwrentius.com

Post reply on HN
Print view View original

91–100 of 174 posts

Re: I think you might be fooling yourself with AI

#91

Post by Aurornis »

podgorniy wrote: post #52
aogaili wrote: post #9

These articles are so off from reality to the point that it can't be taken seriously at all. I keep wondering how they keep popping at HN. At least the author admit he is biased..

> These articles are so off from reality Which ones? That there is no evidence (not personal anecdotes but the numbers) of productivity increase? Or that our subscriptions are heavy subsidized (then try to compare produced value against real price of tokens)? Or that AI companies are burning tonn of cash without clear plan for monetization? Article has good points which are worth discussing rather than discarding. We…

> That there is no evidence (not personal anecdotes but the numbers) of productivity increase?

The blog tries to cite a 2025 METR study as evidence of no productivity increase, but they ignored the newer 2026 study by the same group that did show a productivity increase with the newer tools.

I also think it's funny that there have become these hard demands for studies and proof of increased productivity. We never saw the same standard of evidence applied to previous advancements like different programming languages or using git for version control. We don't argue with people when they say they're personally more productive in emacs than in VS Code or vice versa. It's only when the topic of AI comes up that the bar for evidence gets raised to the sky

Re: I think you might be fooling yourself with AI

#92

Post by palmotea »

somenameforme wrote: post #62
aogaili wrote: post #9

These articles are so off from reality to the point that it can't be taken seriously at all. I keep wondering how they keep popping at HN. At least the author admit he is biased..

I think there's a simple dichotomy based on use cases. For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear. But for things you aren't good at, they're zomg amazing. So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey wha…

> So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey what's all the hype about?' But if you're doing things outside your domain then it's a revolutionary game-changer.

They may also be seen as a revolutionary game-changer by people who actually kinda suck at their job.

And a lot of software engineers actually kinda suck at their jobs.

Re: I think you might be fooling yourself with AI

#93

Post by layer8 »

jimmaswell wrote: post #78
somenameforme wrote: post #62

Earlier quoted context omitted.

I think there's a simple dichotomy based on use cases. For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear. But for things you aren't good at, they're zomg amazing. So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey wha…

> For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear. Preposterous. I can tell 5.6 Sol to do something like "optimize this entire subsystem to have no ongoing memory allocations, fix these 5 bugs, implement these 2 features" and go work on other code or do something else entirely, and it does it all flawlessly. All things I could have done…

> it does it all flawlessly

Serious question: How do you assess this? It seems to me that it would take very significant time to establish that conclusion.

Re: I think you might be fooling yourself with AI

#94

Post by bbg2401 »

jimmaswell wrote: post #78
somenameforme wrote: post #62

Earlier quoted context omitted.

I think there's a simple dichotomy based on use cases. For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear. But for things you aren't good at, they're zomg amazing. So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey wha…

> For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear. Preposterous. I can tell 5.6 Sol to do something like "optimize this entire subsystem to have no ongoing memory allocations, fix these 5 bugs, implement these 2 features" and go work on other code or do something else entirely, and it does it all flawlessly. All things I could have done…

This is going to sound accusatory but I promise it isn’t.

Did you feel the same way about 5.5? The same general sentiment keeps being expressed with every model release. The prior generation is immediately cast aside with a vague “well yes, actually we didn’t mean it last time” attitude.

I think I even heard Theo the T3 guy suggesting that his work comes to halt if he can’t use the latest generation model, despite heaping significant praise on each generation previous and suggesting it does all of his programming tasks.

The boundless energy with which each model generation is said to be revolutionary and leaving everyone behind is tiresome even when incremental progress is real and measurable.

Re: I think you might be fooling yourself with AI

#95

Post by eudamoniac »

sho wrote: post #69
danesparza wrote: post #14

Earlier quoted context omitted.

"These articles are so off from reality to the point that it can't be taken seriously at all" So you disagree with the quote from Feynman? (the quote that forms the basis of the article) I'm genuinely curious to know what part of the article you feel is "so far off from reality"?

There is no longer, in my opinion, any scope for reasonable debate that AI models like Fable 5 are net productivity increases for many, if not most, if not ALL, programming tasks. Anyone can, or should be able to, prove this to themselves in about 5 minutes. And so if someone like the author is trying to argue the contrary, that leads me to believe they either: (1) have not bothered to spend even 5 minutes validating…

Why do you think so many people who have used the same models you do, who are otherwise intelligent, and who should be able to prove to themselves your position in five minutes, don't?

I'm somewhat agnostic about the whole thing, and I find the utter confidence you display strange and arrogant. Some really good devs say AI is not that great. Some other really good devs say it is great. Why is one side of them automatically delusional? Could they know something you don't?

Re: I think you might be fooling yourself with AI

#96

Post by leecommamichael »

krupan wrote: post #70
kalkin wrote: post #34

> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower. METR reran the study early this year and, while they caveat it, this time they found a speedup, which is consistent with subjective estimates of productivity also having increased -- the simplest explanation is that subjective estimates exaggerate, but there'…

"Unfortunately, given participant feedback and surveys, we believe that the data from our new experiment gives us an unreliable signal of the current productivity effect of AI tools." That's more than a caveat. That says, this study is unreliable. And why? "The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without…

Don't let the downvotes get you down. They don't know who funds or constitutes METR. If you actually read these papers, you'll find they cite themselves 4-5 times each publication. They also consider "hosting an HTTP server using Python" to be a "long time horizon" task.

Re: I think you might be fooling yourself with AI

#97

Post by sho »

danesparza wrote: post #87
sho wrote: post #76

Earlier quoted context omitted.

Tell me, when the dot com bubble burst, and all those debt-laden companies collapsed and had buyouts and mergers, was the internet turned off?

Not at all. And to be honest, I don't hear the original author asserting that all AI will go away. But if you have a ChatGPT plan and OpenAI is bought by another company (because they are insolvent) ... do you honestly think you will be able to continue using your service without a significant price increase?

I suppose that would be up to the new owner? How would I know what happens to their services? You shouldn't conflate AI-the-technology with OpenAI-the-company. I'm super bullish about the tech, couldn't care less about a specific company. I mean, I hate Oracle like any other sane person, but that doesn't mean I hate databases.

I'm very confident that AI will continue to be available no matter what happens, at reasonable prices. We don't have to guess - we know what the open source models take to run, the hardware, the electricity, everything. They are served at a profit today - no-one is subsidizing them. Anyone can run GLM5.2 or (soon) K3 etc - you have capex and opex, you can serve X tokens per second, you sell them at Y price, it's just maths.

AI is a technology, not a company or group of companies.

Re: I think you might be fooling yourself with AI

#98

Post by eudamoniac »

Aurornis wrote: post #91
podgorniy wrote: post #52

Earlier quoted context omitted.

> These articles are so off from reality Which ones? That there is no evidence (not personal anecdotes but the numbers) of productivity increase? Or that our subscriptions are heavy subsidized (then try to compare produced value against real price of tokens)? Or that AI companies are burning tonn of cash without clear plan for monetization? Article has good points which are worth discussing rather than discarding. We…

> That there is no evidence (not personal anecdotes but the numbers) of productivity increase? The blog tries to cite a 2025 METR study as evidence of no productivity increase, but they ignored the newer 2026 study by the same group that did show a productivity increase with the newer tools. I also think it's funny that there have become these hard demands for studies and proof of increased productivity. We never saw…

Because those tools were adopted organically by users because they were obviously good. AI is pushed from leaders onto users despite the users not thinking they are good. If my boss decides the whole company is switching to emacs I think he should have some evidence to support that decision, yes.

Re: I think you might be fooling yourself with AI

#99

Post by CollinEMac »

sho wrote: post #69
danesparza wrote: post #14

Earlier quoted context omitted.

"These articles are so off from reality to the point that it can't be taken seriously at all" So you disagree with the quote from Feynman? (the quote that forms the basis of the article) I'm genuinely curious to know what part of the article you feel is "so far off from reality"?

There is no longer, in my opinion, any scope for reasonable debate that AI models like Fable 5 are net productivity increases for many, if not most, if not ALL, programming tasks. Anyone can, or should be able to, prove this to themselves in about 5 minutes. And so if someone like the author is trying to argue the contrary, that leads me to believe they either: (1) have not bothered to spend even 5 minutes validating…

It's pre-Fable 5 but they did site a real study.

Re: I think you might be fooling yourself with AI

#100

Post by eudamoniac »

solumunus wrote: post #71
aogaili wrote: post #9

These articles are so off from reality to the point that it can't be taken seriously at all. I keep wondering how they keep popping at HN. At least the author admit he is biased..

I feel sorry for anyone who isn’t experiencing the gains I’m experiencing. I can see why AI could suck in massive teams, or software with very messy and inconsistent architecture. As a 2 man developer team working on a mono repo with clear and simple architecture, consistent schema conventions, and heavy context available, it’s becoming ludicrous how fast we can pump out stable features. We must be 3-4x faster at thi…

>pump out stable features

>data corruption...urgent fix required

Do you people hear yourselves?

Post reply on HN