Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's be honest, Generative AI isn't going all that well

garymarcus.substack.com

Post reply on HN
Print view View original

41–50 of 346 posts

Re: Let's be honest, Generative AI isn't going all that well

#41

Post by elzbardico »

maccard wrote: post #14
tombert wrote: post #10

I find it a bit odd that people are acting like this stuff is an abject failure because it's not perfect yet. Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving. Yes, people have probably been deploying it in spots where it's not quite ready but it's myopic to act like it's "not going all that well" when it's pretty clear that it actually is g…

> Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving. Every 2/3 months we're hearing there's a new model that just blows the last one out of the water for coding. Meanwhile, here I am with Opus and Sonnet for $20/mo and it's regularly failing at basic tasks, antigravity getting stuck in loops and burning credits. We're talking "copy basic exam…

There’s a subtle point a moment when you HAVE to take the driver wheel from the AI. All issues I see are from people insisting to use far beyond the point it stops being useful.

It is a helper, a partner, it is still not ready go the last mile

Re: Let's be honest, Generative AI isn't going all that well

#42

Post by gejose »

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1]

> In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc.

> In 2029, AI will not be able to read a novel and reliably answer questions about plot, character, conflicts, motivations, etc. Key will be going beyond the literal text, as Davis and I explain in Rebooting AI.

> In 2029, AI will not be able to work as a competent cook in an arbitrary kitchen (extending Steve Wozniak’s cup of coffee benchmark).

> In 2029, AI will not be able to reliably construct bug-free code of more than 10,000 lines from natural language specification or by interactions with a non-expert user. [Gluing together code from existing libraries doesn’t count.]

> In 2029, AI will not be able to take arbitrary proofs from the mathematical literature written in natural language and convert them into a symbolic form suitable for symbolic verification.

Many of these have already been achieved, and it's only early 2026.

[1]https://garymarcus.substack.com/p/dear-elon-musk-here-are-fi...

Re: Let's be honest, Generative AI isn't going all that well

#43

Post by tombert »

drewbug01 wrote: post #32
BeetleB wrote: post #16

Earlier quoted context omitted.

> We're talking "copy basic examples and don't hallucinate APIs" here, not deep complicated system design topics. If your metric is an LLM that can copy/paste without alterations, and never hallucinate APIs, then yeah, you'll always be disappointed with them. The rest of us learn how to be productive with them despite these problems.

> If your metric is an LLM that can copy/paste without alterations, and never hallucinate APIs, then yeah, you'll always be disappointed with them. I struggle to take comments like this seriously - yes, it is very reasonable to expect these magical tools to copy and paste something without alterations. How on earth is that an unreasonable ask? The whole discourse around LLMs is so utterly exhausting. If I say I don't…

I think what they're best at right now is the initial scaffolding work of projects. A lot of the annoying bootstrap shit that I hate doing is actually generally handled really well by Codex.

I agree that there's definitely some overhype to them right now. At least for the stuff I've done they have gotten considerably better though, to a point where the code it generates is often usable, if sub-optimal.

For example, about three years ago, I was trying to get ChatGPT to write me a C program to do a fairly basic ZeroMQ program. It generated something that looked correct, but it would crash pretty much immediately, because it kept trying to use a pointer after free.

I tried the same thing again with Codex about a week ago, and it worked out of the box, and I was even able to get it to do more stuff.

Re: Let's be honest, Generative AI isn't going all that well

#44

Post by herunan »

First of all, popping in a few screenshots of articles and papers is not proper analysis.

Second of all, GenAI is going well or not depending on how we frame it.

In terms of saving time, money and effort when coding, writing, analysing, researching, etc. It’s extremely successful.

In terms of leading us to AGI… GenAI alone won’t reach that. Current ROI is plateauing, and we need to start investing more somewhere else.

Re: Let's be honest, Generative AI isn't going all that well

#45

Post by nsoonhui »

mattmaroon wrote: post #6

Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. I myself am saving a small fortune on design and photography and getting better results while doing it. If this is not all that well I can’t wait until we get to mediocre!

It's not directly comparable. The first time writing the code is always the hardest because you might have to figure out the requirements along the way. When you have the initial system running for a while, doing a second one is easier because all the requirements kinks are figured out.

By the way, why does your co-founder have to do the rewrite at all?

Re: Let's be honest, Generative AI isn't going all that well

#46

Post by Aurornis »

maccard wrote: post #14
tombert wrote: post #10

I find it a bit odd that people are acting like this stuff is an abject failure because it's not perfect yet. Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving. Yes, people have probably been deploying it in spots where it's not quite ready but it's myopic to act like it's "not going all that well" when it's pretty clear that it actually is g…

> Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving. Every 2/3 months we're hearing there's a new model that just blows the last one out of the water for coding. Meanwhile, here I am with Opus and Sonnet for $20/mo and it's regularly failing at basic tasks, antigravity getting stuck in loops and burning credits. We're talking "copy basic exam…

You’re doing exactly the thing that the parent commenter pointed out: Complaining that they’re not perfect yet as if that’s damning evidence of failure.

We all know LLMs get stuck. We know they hallucinate. We know they get things wrong. We know they get stuck in loops.

There are two types of people: The first group learns to work within these limits and adapt to using them where they’re helpful while writing the code when they’re not.

The second group gets frustrated every time it doesn’t one-shot their prompt and declares it all a big farce. Meanwhile the rest of us are out here having fun with these tools, however limited they are.

Re: Let's be honest, Generative AI isn't going all that well

#47

Post by rpowers »

I keep reading comments that claim GenAI's positive traits, but this usually amounts to some toy PoC that very eerily mirrors work found in code bootcamps. You want an app that has logins and comments and upvotes? GenAI is going to look amazing setting up a non-relational db to your node backend.

Re: Let's be honest, Generative AI isn't going all that well

#50

Post by bwestergard »

mattmaroon wrote: post #6

Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. I myself am saving a small fortune on design and photography and getting better results while doing it. If this is not all that well I can’t wait until we get to mediocre!

Out of curiosity, what is your product?
Post reply on HN