Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's be honest, Generative AI isn't going all that well

garymarcus.substack.com

Post reply on HN
Print view View original

151–160 of 346 posts

Re: Let's be honest, Generative AI isn't going all that well

#151

Post by raincole »

jgalt212 wrote: post #104
gejose wrote: post #42

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…

This comment or something very close always appears alongside a Gary Marcus post.

And why not? Is there any reason for this comment to not appear?

If Bill Gates made a predication about computing, no matter what the predication says, you can bet that 640K memory quote would be mentioned in the comment section (even he didn't actually say that).

Re: Let's be honest, Generative AI isn't going all that well

#152

Post by dyauspitr »

gejose wrote: post #42

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…

In my opinion, contrary to other comments here I think AI can do all of the above already except being a kitchen cook.

Just earlier today I asked it to give me a summary of a show I was watching until a particular episode in a particular season without spoiling the rest of it and it did a great job.

Re: Let's be honest, Generative AI isn't going all that well

#153

Post by siscia »

I think that the wider industry is living right now what was coding and software engineering around 1 year or so ago.

Yeah you could ask ChatGPT or Claude to write code, but it wasn't really there.

It needs a while to adopt the model AND the UI. As in software are the first one because we are both makers and users.

Re: Let's be honest, Generative AI isn't going all that well

#154

Post by [deleted] »

stingrae wrote: post #85
merlincorey wrote: post #59

Earlier quoted context omitted.

Which ones are you claiming have already been achieved? My understanding of the current scorecard is that he's still technically correct, though I agree with you there is velocity heading towards some of these things being proven wrong by 2029. For example, in the recent thread about LLMs and solving an Erdos problem I remember reading in the comments that it was confirmed there were multiple LLMs involved as well as…

1 and 2 have been achieved. 4 is close, the interface needs some work to allow nontechnical people use it. (claude code)

[deleted]

Re: Let's be honest, Generative AI isn't going all that well

#155

Post by littlestymaar »

colechristensen wrote: post #87
zozbot234 wrote: post #60

Earlier quoted context omitted.

> In 2029, AI will not be able to read a novel and reliably answer questions about plot, character, conflicts, motivations, etc. Key will be going beyond the literal text, as Davis and I explain in Rebooting AI. Can AI actually do this? This looks like a nice benchmark for complex language processing, since a complete novel takes up a whole lot of context (consider War and Peace or The Count of Monte Cristo ). Of cou…

>Can AI actually do this? This looks like a nice benchmark for complex language processing, since a complete novel takes up a whole lot of context (consider War and Peace or The Count of Monte Cristo) Yes, you just break the book down by chapters or whatever conveniently fits in the context window to produce summaries such that all of the chapter summaries can fit in one context window. You could also do something wi…

> Yes, you just break the book down by chapters or whatever conveniently fits in the context window to produce summaries such that all of the chapter summaries can fit in one context window.

I've done that a few month ago and in fact doing just this will miss cross-chapter informations (say something is said in chapter 1, that doesn't appears to be important but reveals itself crucial later on, like "Chekhov's gun").

Maybe doing that iteratively several time would solve the problem, I run out of time and didn't try but the straightforward workflow you're describing doesn't work so I think it's fair to say this challenge isn't solve. (It works better with non-fiction though, because the prose is usually drier and straight to the point).

Re: Let's be honest, Generative AI isn't going all that well

#156

Post by cameronh90 »

1a527dd5 wrote: post #21

A year ago I would have agreed wholeheartedly and I was a self confessed skeptic. Then Gemini got good (around 2.5?), like I-turned-my-head good. I started to use it every week-ish, not to write code. But more like a tool (as you would a calculator). More recently Opus 4.5 was released and now I'm using it every day to assist in code. It is regularly helping me take tasks that would have taken 6-12 hours down to 15-3…

I'm now putting more queries into LLMs than I am into Google Search.

I'm not sure how much of that is because Google Search has worsened versus LLMs having improved, but it's still a substantial shift in my day-to-day life.

Something like finding the most appropriate sensor ICs to use for a particular use case requires so much less effort than it used to. I might have spent an entire day digging through data sheets before, and now I'll find what I need in a few minutes. It feels at least as revolutionary as when search replaced manually paging through web directories.

Re: Let's be honest, Generative AI isn't going all that well

#158

Post by wombat-man »

aprdm wrote: post #26
mattmaroon wrote: post #6

Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. I myself am saving a small fortune on design and photography and getting better results while doing it. If this is not all that well I can’t wait until we get to mediocre!

lol same. I just wrote a bunch of diagrams with mermaid that would legit take me a week, also did a mock of an UI for a frontend engineer that would take me another week to do .. or some designers. All of that in between meetings... Waiting for it to actually go well to see what else I can do !

I have been able to prototype way faster. I can explain how I want a prototype reworked and it's often successful. Doesn't always work, but super useful more often than not.

Re: Let's be honest, Generative AI isn't going all that well

#159

Post by pixl97 »

fxtentacle wrote: post #88
stingrae wrote: post #85

Earlier quoted context omitted.

1 and 2 have been achieved. 4 is close, the interface needs some work to allow nontechnical people use it. (claude code)

I strongly disagree. I’ve yet to find an AI that can reliably summarise emails, let alone understand nuance or sarcasm. And I just asked ChatGPT 5.2 to describe an Instagram image. It didn’t even get the easily OCR-able text correct. Plus it completely failed to mention anything sports or stadium related. But it was looking at a cliche baseball photo taken by an fan inside the stadium.

>let alone understand nuance or sarcasm

I'm still trying to find humans that do this reliably too.

To add on, 5.2 seems to be kind of lazy when reading text in images by default. Feeding it an image it may give the first word or so. But coming back with a prompt 'read all the text in the image' makes it do a better job.

With one in particular that I tested I thought it was hallucinating some of the words, but there was a picture in the picture with small words it saw I missed the first time.

I think a lot of AI capabilities are kind of munged to end users because they limit how much GPU is used.

Re: Let's be honest, Generative AI isn't going all that well

#160

Post by BeetleB »

maccard wrote: post #109
tombert wrote: post #22

Earlier quoted context omitted.

Sure, but think about what it's replacing. If you hired a human, it will cost you thousands a week. Humans will also fail at basic tasks, get stuck in useless loops, and you still have to pay them for all that time. For that matter, even if I'm not hiring anyone, I will still get stuck on projects and burn through the finite number of hours I have on this planet trying to figure stuff out and being wrong for a lot of…

You’ve missed my point here - I agree that gen AI has changed everything and is useful, _but_ I disagree that it’s improved substantially - which is what the comment I replied to claimed. Anecdotally I’ve seen no difference in model changes in the last year, but going from LLM to Claude code (where we told the LLMs they can use tools on our machines) was a game changer. The improvement there was the agent loop and th…

I've been coding with LLMs for less than a year. As I mentioned to someone in email a few days ago: In the first half, when an LLM solved a problem differently from me, I would probe why and more often than not overrule and instruct it to do it my way.

Now it's reversed. More often than not its method is better than mine (e.g. leveraging a better function/library than I would have).

In general, it's writing idiomatic mode much more often. It's been many months since I had to correct it and tell it to be idiomatic.

Post reply on HN