Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's be honest, Generative AI isn't going all that well

garymarcus.substack.com

Post reply on HN
Print view View original

51–60 of 346 posts

Re: Let's be honest, Generative AI isn't going all that well

#51

Post by Aurornis »

dreadsword wrote: post #18

This feels like a pretty low effort post that plays heavily to superficial reader's cognitive biases. I work commercializing AI in some very specific use cases where it extremely valuable. Where people are being lead astray is layering generalizations: general use cases (copilots) deployed across general populations and generally not doing very well. But that's PMF stuff, not a failure of the underlying tech.

> This feels like a pretty low effort post that plays heavily to superficial reader's cognitive biases.

I haven’t followed this author but the few times he’s come up his writings have been exactly this.

Re: Let's be honest, Generative AI isn't going all that well

#52

Post by blibble »

drewbug01 wrote: post #32
BeetleB wrote: post #16

Earlier quoted context omitted.

> We're talking "copy basic examples and don't hallucinate APIs" here, not deep complicated system design topics. If your metric is an LLM that can copy/paste without alterations, and never hallucinate APIs, then yeah, you'll always be disappointed with them. The rest of us learn how to be productive with them despite these problems.

> If your metric is an LLM that can copy/paste without alterations, and never hallucinate APIs, then yeah, you'll always be disappointed with them. I struggle to take comments like this seriously - yes, it is very reasonable to expect these magical tools to copy and paste something without alterations. How on earth is that an unreasonable ask? The whole discourse around LLMs is so utterly exhausting. If I say I don't…

> What, precisely, are they good for?

scamming people

Re: Let's be honest, Generative AI isn't going all that well

#53

Post by falloutx »

tombert wrote: post #22
maccard wrote: post #14

Earlier quoted context omitted.

> Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving. Every 2/3 months we're hearing there's a new model that just blows the last one out of the water for coding. Meanwhile, here I am with Opus and Sonnet for $20/mo and it's regularly failing at basic tasks, antigravity getting stuck in loops and burning credits. We're talking "copy basic exam…

Sure, but think about what it's replacing. If you hired a human, it will cost you thousands a week. Humans will also fail at basic tasks, get stuck in useless loops, and you still have to pay them for all that time. For that matter, even if I'm not hiring anyone, I will still get stuck on projects and burn through the finite number of hours I have on this planet trying to figure stuff out and being wrong for a lot of…

I am an AI-skeptic but I would agree this looks impressive from certain angles, especially if you're an early startup (maybe) or you are very high up the chain and just want to focus on cutting costs. On the other hand, if you are about to be unemployed, this is less impressive. Can it replace a human? I would say no its still long way to go, but a good salesman can convince executives that it does and thats all that matters.

Re: Let's be honest, Generative AI isn't going all that well

#54

Post by venndeezl »

mattmaroon wrote: post #6

Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. I myself am saving a small fortune on design and photography and getting better results while doing it. If this is not all that well I can’t wait until we get to mediocre!

I suspect he means as a trillion dollar corporation led endeavor.

I trained a small neural net on pics of a cat I had in the 00s (RIP George, you were a good cat).

Mounted a webcam I had gotten for free from somewhere, above the cat door, in the exterior of the house.

If the neural net recognized my cat it switched off an electromagnetic holding the pet door locked. Worked perfectly until I moved out of the rental.

Neural nets are, end of the day, pretty cool. It's the data center business that's the problem. Just more landlords, wannabe oligarchs, claiming ownership over anything they can get the politicians to give them.

Re: Let's be honest, Generative AI isn't going all that well

#55

Post by hactually »

fzeroracer wrote: post #33
mattmaroon wrote: post #6

Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. I myself am saving a small fortune on design and photography and getting better results while doing it. If this is not all that well I can’t wait until we get to mediocre!

> Meanwhile, my cofounder is rewriting code we spent millions of salary on in the past by himself in a few weeks. This is one of those statements that would horrify any halfway competent engineer. A cowboy coder going in, seeing a bunch of code and going 'I should rewrite this' is one of the biggest liabilities to any stable system.

I assume this is because they're already insanely profitable after hitting PMF and are now trying to bring down infra costs?

Right? RIGHT?!

Re: Let's be honest, Generative AI isn't going all that well

#56

Post by tombert »

falloutx wrote: post #53
tombert wrote: post #22

Earlier quoted context omitted.

Sure, but think about what it's replacing. If you hired a human, it will cost you thousands a week. Humans will also fail at basic tasks, get stuck in useless loops, and you still have to pay them for all that time. For that matter, even if I'm not hiring anyone, I will still get stuck on projects and burn through the finite number of hours I have on this planet trying to figure stuff out and being wrong for a lot of…

I am an AI-skeptic but I would agree this looks impressive from certain angles, especially if you're an early startup (maybe) or you are very high up the chain and just want to focus on cutting costs. On the other hand, if you are about to be unemployed, this is less impressive. Can it replace a human? I would say no its still long way to go, but a good salesman can convince executives that it does and thats all that…

I just think Jevins paradox [1]/Gustafson's Law [2] kind of applies here.

Maybe I shouldn't have used the word "replaced", as I don't really think it's actually going to "replace" people long term. I think it's likely to just lead to higher output as these get better and better .

[1] https://en.wikipedia.org/wiki/Jevons_paradox

[2] https://en.wikipedia.org/wiki/Gustafson%27s_law

Re: Let's be honest, Generative AI isn't going all that well

#57

Post by barbazoo »

tombert wrote: post #10

I find it a bit odd that people are acting like this stuff is an abject failure because it's not perfect yet. Generative AI, as we know it, has only existed ~5-6 years, and it has improved substantially, and is likely to keep improving. Yes, people have probably been deploying it in spots where it's not quite ready but it's myopic to act like it's "not going all that well" when it's pretty clear that it actually is g…

We implement pretty cool workflows at work using "GenAI" and the users of our software are really appreciative. It's like saying a hammer sucks because it breaks most things you hit with it.

Re: Let's be honest, Generative AI isn't going all that well

#58

Post by ls612 »

gejose wrote: post #42

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…

I'm pretty sure it can do all of those except for the one which requires a physical body (in the kitchen) and the one that humans can't do reliably either (construct 10000 loc bug-free).

Re: Let's be honest, Generative AI isn't going all that well

#59

Post by merlincorey »

gejose wrote: post #42

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…

Which ones are you claiming have already been achieved?

My understanding of the current scorecard is that he's still technically correct, though I agree with you there is velocity heading towards some of these things being proven wrong by 2029.

For example, in the recent thread about LLMs and solving an Erdos problem I remember reading in the comments that it was confirmed there were multiple LLMs involved as well as an expert mathematician who was deciding what context to shuttle between them and helping formulate things.

Similarly, I've not yet heard of any non-expert Software Engineers creating 10,000+ lines of non-glue code that is bug-free. Even expert Engineers at Cloud Flare failed to create a bug-free OAuth library with Claude at the helm because some things are just extremely difficult to create without bugs even with experts in the loop.

Re: Let's be honest, Generative AI isn't going all that well

#60

Post by zozbot234 »

gejose wrote: post #42

I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…

> In 2029, AI will not be able to read a novel and reliably answer questions about plot, character, conflicts, motivations, etc. Key will be going beyond the literal text, as Davis and I explain in Rebooting AI.

Can AI actually do this? This looks like a nice benchmark for complex language processing, since a complete novel takes up a whole lot of context (consider War and Peace or The Count of Monte Cristo). Of course the movie variety is even more challenging since it involves especially complex multi-modal input. You could easily extend it to making sense of a whole TV series.

Post reply on HN