Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's be honest, Generative AI isn't going all that well

garymarcus.substack.com

Post reply on HN
Print view View original

321–330 of 346 posts

Re: Let's be honest, Generative AI isn't going all that well

#321

Post by solumunus »

immibis wrote: post #246
serf wrote: post #190

Earlier quoted context omitted.

it's wild that somehow with regards to AI conversations lately someone can say "I saved 3 months doing X" and someone can willfully and thoughtfully reply "No you didn't , you're wrong." without hesitation. I feel bad for AI opponents mostly because it seems like the drive to be against the thing is stronger than the drive towards fact or even kindness. My .02c: I am saving months of efforts using AI tools to fix old…

Just look at the METR study. All predictions were massive time savings but all observations were massive time losses. That's why we don't believe you when you say you saved time.

You should know better than to form a opinion from one study. I could show you endless examples of a study concluding untrue things, endless…

I’ve been full time (almost solo) building an ERP system for years and my development velocity has gone roughly 2x. The new features are of equal quality, everything is code reviewed, everything is done in my style, adhering to my architectural patterns. Not to mention I’ve managed to build a mobile app alongside my normal full time work, something I wouldn’t have even had the time to attempt to learn about without the use of agents.

So do you think I’m lying or do you just think my eyes are deceiving me somehow?

Re: Let's be honest, Generative AI isn't going all that well

#322

Post by Dylan16807 »

solumunus wrote: post #321
immibis wrote: post #246

Earlier quoted context omitted.

Just look at the METR study. All predictions were massive time savings but all observations were massive time losses. That's why we don't believe you when you say you saved time.

You should know better than to form a opinion from one study. I could show you endless examples of a study concluding untrue things, endless… I’ve been full time (almost solo) building an ERP system for years and my development velocity has gone roughly 2x. The new features are of equal quality, everything is code reviewed, everything is done in my style, adhering to my architectural patterns. Not to mention I’ve man…

I think any measurement of development velocity is shaky, especially when measured between two different workflows, and especially when measured by the person doing the development.

Such an estimate is far less reliable than your eyes are.

So if people want to do more and better studies, that sounds great. But I have a good supply of salt for self-estimates. I'm listening to your input, but it's much easier for your self-assessment to have issues than you're implying.

Re: Let's be honest, Generative AI isn't going all that well

#323

Post by Dylan16807 »

mattmaroon wrote: post #242
idiotsecant wrote: post #185

Earlier quoted context omitted.

I agree that code is being written in exactly the same sense that Teslas are driving themselves.

Yes, in both cases it takes someone to steer. It’s not a complete solution. Someone who can’t drive can’t just ride a Tesla around town and someone who can’t program can’t vibe code anything complex. But if it can do 90% of the work for you, it is a serious force magnifier.

> But if it can do 90% of the work for you, it is a serious force magnifier.

Well, we could characterize a Tesla as doing 90% of the work but it's not at all a force multiplier. Your "10%" supervisory contribution takes just as long as doing 100%.

Re: Let's be honest, Generative AI isn't going all that well

#324

Post by zombot »

tombert wrote: post #22
maccard wrote: post #14

Earlier quoted context omitted.

> Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving. Every 2/3 months we're hearing there's a new model that just blows the last one out of the water for coding. Meanwhile, here I am with Opus and Sonnet for $20/mo and it's regularly failing at basic tasks, antigravity getting stuck in loops and burning credits. We're talking "copy basic exam…

Sure, but think about what it's replacing. If you hired a human, it will cost you thousands a week. Humans will also fail at basic tasks, get stuck in useless loops, and you still have to pay them for all that time. For that matter, even if I'm not hiring anyone, I will still get stuck on projects and burn through the finite number of hours I have on this planet trying to figure stuff out and being wrong for a lot of…

Making humans look ridiculously and unrealisticly bad still doesn't invalidate criticism of AI, its overhyped marketing and all the astroturfing.

Re: Let's be honest, Generative AI isn't going all that well

#325

Post by [deleted] »

solumunus wrote: post #321
immibis wrote: post #246

Earlier quoted context omitted.

Just look at the METR study. All predictions were massive time savings but all observations were massive time losses. That's why we don't believe you when you say you saved time.

You should know better than to form a opinion from one study. I could show you endless examples of a study concluding untrue things, endless… I’ve been full time (almost solo) building an ERP system for years and my development velocity has gone roughly 2x. The new features are of equal quality, everything is code reviewed, everything is done in my style, adhering to my architectural patterns. Not to mention I’ve man…

[deleted]

Re: Let's be honest, Generative AI isn't going all that well

#326

Post by fuy »

solumunus wrote: post #321
immibis wrote: post #246

Earlier quoted context omitted.

Just look at the METR study. All predictions were massive time savings but all observations were massive time losses. That's why we don't believe you when you say you saved time.

You should know better than to form a opinion from one study. I could show you endless examples of a study concluding untrue things, endless… I’ve been full time (almost solo) building an ERP system for years and my development velocity has gone roughly 2x. The new features are of equal quality, everything is code reviewed, everything is done in my style, adhering to my architectural patterns. Not to mention I’ve man…

Not saying you're wrong, but solo developers building (relatively) greenfield projects are the best bet for increased AI productivity.

Solo dev projects are usually reasonably sized (Good studies look at a broader picture.

Re: Let's be honest, Generative AI isn't going all that well

#327

Post by solumunus »

fuy wrote: post #326
solumunus wrote: post #321

Earlier quoted context omitted.

You should know better than to form a opinion from one study. I could show you endless examples of a study concluding untrue things, endless… I’ve been full time (almost solo) building an ERP system for years and my development velocity has gone roughly 2x. The new features are of equal quality, everything is code reviewed, everything is done in my style, adhering to my architectural patterns. Not to mention I’ve man…

Not saying you're wrong, but solo developers building (relatively) greenfield projects are the best bet for increased AI productivity. Solo dev projects are usually reasonably sized ( Good studies look at a broader picture.

It’s a very good point. I have full control and everything is incredibly uniform, and more recently designed with agents in mind. This must make things significantly easier for the LLM.

Re: Let's be honest, Generative AI isn't going all that well

#328

Post by mattmaroon »

kunley wrote: post #258
mattmaroon wrote: post #167

Earlier quoted context omitted.

The codebase is old and really hard to work on. It’s a game that existed pre-iPhone and still has decent revenue but could use some updating. We intentionally shrank our company down to auto-pilot mode and frankly don’t even have a working development environment anymore. It was basically cost prohibitive to change anything significant until Claude became able to do most of the work for us. My cofounder (also CTO of…

Millions? Of people in self driving teslas? The actual number of such vehicles produced is two orders of magnitude less.

No it isn’t. They sold 1.6 million vehicles last year and every one of them can self drive. At least once a year they give out free access to it and I don’t know a Tesla owner who hasn’t tried it.

You’re right that it doesn’t happen every day but that doesn’t change the point, you all are debating whether something can happen after it already happened.

Re: Let's be honest, Generative AI isn't going all that well

#329

Post by jbbryant »

mattmaroon wrote: post #6

Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. I myself am saving a small fortune on design and photography and getting better results while doing it. If this is not all that well I can’t wait until we get to mediocre!

Exactly, our venture studio that partnered with our startup collapsed. The code there team wrote was that took two years was terrible and didn't fully function. My CTO and I are rewriting 60% of the code with AI! Now everything works with bugs.....

Re: Let's be honest, Generative AI isn't going all that well

#330

Post by staticman2 »

gejose wrote: post #42

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…

I don't understand how this claim can even be tested:

> In 2029, AI will not be able to read a novel and reliably answer questions about plot, character, conflicts, motivations, etc. Key will be going beyond the literal text, as Davis and I explain in Rebooting AI.

Once you are "going beyond the literal text" the standard is usefulness of your insight about the novel, not whether your insight is "right" or "wrong".

Post reply on HN