Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's be honest, Generative AI isn't going all that well

garymarcus.substack.com

Post reply on HN
Print view View original

81–90 of 346 posts

Re: Let's be honest, Generative AI isn't going all that well

#81

Post by wouldbecouldbe »

merlincorey wrote: post #25
mattmaroon wrote: post #6

Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. I myself am saving a small fortune on design and photography and getting better results while doing it. If this is not all that well I can’t wait until we get to mediocre!

> Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. Code is not an asset it's a liability, and code that no one has reviewed is even more of a liability. However, in the end, execution is all that matters so if you and your cofounder are able to execute successfully with mountains of generated code then it doesn't matter what assets and liabilities you hol…

Developers that can’t see the change are blind.

Just this week, sun-tue. I added a fully functional subscription model to an existing platform, build out a bulk async elasticjs indexing for a huge database and migrated a very large Wordpress website to NextJS. 2.5 days, would have cost me at least a month 2 years ago.

Re: Let's be honest, Generative AI isn't going all that well

#82

Post by thethirdone »

kleene_op wrote: post #79
thethirdone wrote: post #63

Earlier quoted context omitted.

Which ones of those have been achieved in your opinion? I think the arbitrary proofs from mathematical literature is probably the most solved one. Research into IMO problems, and Lean formalization work have been pretty successful. Then, probably reading a novel and answering questions is the next most successful. Reliably constructing 10k bug free lines is probably the least successful. AI tends to produce more bugs…

> Reliably constructing 10k bug free lines is probably the least successful. You imperatively need to try Claude Code, because it absolutely does that.

I have seen many people try to use Claude Code and get LOTS of bugs. Show me any > 10k project you have made with it and I will put the effort in to find one bug free of charge.

Re: Let's be honest, Generative AI isn't going all that well

#83

Post by onlyrealcuzzo »

tombert wrote: post #10

I find it a bit odd that people are acting like this stuff is an abject failure because it's not perfect yet. Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving. Yes, people have probably been deploying it in spots where it's not quite ready but it's myopic to act like it's "not going all that well" when it's pretty clear that it actually is g…

> Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving.

I think the big problem is that the pace of improvement was UNBELIEVABLE for about 4 years, and it appears to have plateaued to almost nothing.

ChatGPT has barely improved in, what, 6 months or so.

They are driving costs down incredibly, which is not nothing.

But, here's the thing, they're not cutting costs because they have to. Google has deep enough pockets.

They're cutting costs because - at least with the current known paradigm - the cost is not worth it to make material improvements.

So unless there's a paradigm shift, we're not seeing MASSIVE improvements in output like we did in the previous years.

You could see costs go down to 1/100th over 3 years, seriously.

But they need to make money, so it's possible non of that will be passed on.

Re: Let's be honest, Generative AI isn't going all that well

#84

Post by acdha »

hamdingers wrote: post #78
jbs789 wrote: post #20

Earlier quoted context omitted.

Because the likes of Altman have set short term expectations unrealistically high.

I maintain that most anti-AI sentiment is actually anti-lying-tech-CEO sentiment misattributed. The technology is neat, the people selling it are ghouls.

Exactly: the technology is useful but because the executive class is hyping it as close to AGI because their buddies are slavering for layoffs. If that “when do you get fired?” tone wasn’t behind the conversation, I think a lot of people would be interested in applying LLMs to the smaller subset of things they actually perform well at.

Re: Let's be honest, Generative AI isn't going all that well

#85

Post by stingrae »

merlincorey wrote: post #59
gejose wrote: post #42

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…

Which ones are you claiming have already been achieved? My understanding of the current scorecard is that he's still technically correct, though I agree with you there is velocity heading towards some of these things being proven wrong by 2029. For example, in the recent thread about LLMs and solving an Erdos problem I remember reading in the comments that it was confirmed there were multiple LLMs involved as well as…

1 and 2 have been achieved.

4 is close, the interface needs some work to allow nontechnical people use it. (claude code)

Re: Let's be honest, Generative AI isn't going all that well

#86

Post by falloutx »

drewbug01 wrote: post #32
BeetleB wrote: post #16

Earlier quoted context omitted.

> We're talking "copy basic examples and don't hallucinate APIs" here, not deep complicated system design topics. If your metric is an LLM that can copy/paste without alterations, and never hallucinate APIs, then yeah, you'll always be disappointed with them. The rest of us learn how to be productive with them despite these problems.

> If your metric is an LLM that can copy/paste without alterations, and never hallucinate APIs, then yeah, you'll always be disappointed with them. I struggle to take comments like this seriously - yes, it is very reasonable to expect these magical tools to copy and paste something without alterations. How on earth is that an unreasonable ask? The whole discourse around LLMs is so utterly exhausting. If I say I don't…

Its strong enough to replace humans at their jobs and weak enough that it cant do basic things. Its a paradox. Just learn to be productive with them. Pay $200/month and work around with its little quirks. /s

Re: Let's be honest, Generative AI isn't going all that well

#87

Post by colechristensen »

zozbot234 wrote: post #60
gejose wrote: post #42

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…

> In 2029, AI will not be able to read a novel and reliably answer questions about plot, character, conflicts, motivations, etc. Key will be going beyond the literal text, as Davis and I explain in Rebooting AI. Can AI actually do this? This looks like a nice benchmark for complex language processing, since a complete novel takes up a whole lot of context (consider War and Peace or The Count of Monte Cristo ). Of cou…

>Can AI actually do this? This looks like a nice benchmark for complex language processing, since a complete novel takes up a whole lot of context (consider War and Peace or The Count of Monte Cristo)

Yes, you just break the book down by chapters or whatever conveniently fits in the context window to produce summaries such that all of the chapter summaries can fit in one context window.

You could also do something with a multi-pass strategy where you come up with a collection of ideas on the first pass and then look back with search to refine and prove/disprove them.

Of course for novels which existed before the time of training an LLM will already contain trained information about so having it "read" classic works like The Count of Monte Cristo and answer questions about it would be a bit of an unfair pass of the test because models will be expected to have been trained on large volumes of existing text analysis on that book.

>reliably answer questions about plot, character, conflicts, motivations

LLMs can already do this automatically with my code in a sizable project (you know what I mean), it seems pretty simple to get them to do it with a book.

Re: Let's be honest, Generative AI isn't going all that well

#88

Post by fxtentacle »

stingrae wrote: post #85
merlincorey wrote: post #59

Earlier quoted context omitted.

Which ones are you claiming have already been achieved? My understanding of the current scorecard is that he's still technically correct, though I agree with you there is velocity heading towards some of these things being proven wrong by 2029. For example, in the recent thread about LLMs and solving an Erdos problem I remember reading in the comments that it was confirmed there were multiple LLMs involved as well as…

1 and 2 have been achieved. 4 is close, the interface needs some work to allow nontechnical people use it. (claude code)

I strongly disagree. I’ve yet to find an AI that can reliably summarise emails, let alone understand nuance or sarcasm. And I just asked ChatGPT 5.2 to describe an Instagram image. It didn’t even get the easily OCR-able text correct. Plus it completely failed to mention anything sports or stadium related. But it was looking at a cliche baseball photo taken by an fan inside the stadium.

Re: Let's be honest, Generative AI isn't going all that well

#90

Post by falloutx »

stingrae wrote: post #85
merlincorey wrote: post #59

Earlier quoted context omitted.

Which ones are you claiming have already been achieved? My understanding of the current scorecard is that he's still technically correct, though I agree with you there is velocity heading towards some of these things being proven wrong by 2029. For example, in the recent thread about LLMs and solving an Erdos problem I remember reading in the comments that it was confirmed there were multiple LLMs involved as well as…

1 and 2 have been achieved. 4 is close, the interface needs some work to allow nontechnical people use it. (claude code)

I dispute 1 & 2 more than 4.

1) Is it actually watching a movie frame by frame or just searching about it and then giving you the answer?

2) Again can it handle very long novels, context windows are limited and it can easily miss something. Where is the proof for this?

4 is probably solved

4) This is more on predictor because this is easy to game. you can create some gibberish code with LLM today that is 10k lines long without issues. Even a non-technical user can do

Post reply on HN