Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

web.archive.org

Post reply on HN
Print view View original

11–20 of 52 posts

Re: Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

#11

Post by aurareturn »

dekhn wrote: post #5

Except for a short window around the release of GPT-4 (especially the inflated claims around beating expert trained humans at legal and math tests, as well as "replacing google"), I think people have more or less right-sized their expectations for large language models and generative AI. Clearly it can do interesting and impressive things but it's not superintelligence, and the folks predicting we're just around the…

I don't think anyone thought it was super intelligence.

I think it's impressive that we went from LLMs not being useful at all to GPT3.5 shocking the world to GPT4 becoming super useful for many things in around 7 months time.

LLM progress have slowed down a bit. But I think we're just getting started. It's still really early. It's only been 1 year since GPT4 came out. Even at the level of GPT4, scaling it would have immense benefits. But my sense is that we'll have a few more levels of great leaps in LLM capabilities that will shock people in the next 3-4 years.

Re: Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

#13

Post by weweweoo »

Generative AI appears fantastic aid for many smaller tasks where there's enough training data, and correctness of the answer is subjective (like art), or easily verifiable by a human in the loop (small snippets of code, checking that summary of an article matches the contents of the original). Generally it helps with the tedious parts, but not with the hard parts of my job.

I don't have much belief in fully autonomous generative AI agents performing more complex tasks any time soon. It's a significant productivity boost for some jobs, but not a total replacement for humans who do more than read from a script, or write clickbait articles for media.

Re: Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

#14

Post by [deleted] »

echelon wrote: post #10
zzzbra wrote: post #8

Heartbreaking: The Worst People You Know Just Made A Great Point

The music, film, and game industries are about to be completely disrupted. LLMs and AGI might be hogwash, but processing multimedia is where Gen AI and especially diffusion models shine. Furthermore text-to-{whatever} models might produce slop, but Gen AI "exoskeletons" (spatial domain, temporal domain editors) are Photoshop and Blender from next century. These turbocharge creatives. Hearing and vision are simple ope…

[deleted]

Re: Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

#16

Post by torginus »

karaterobot wrote: post #9
dekhn wrote: post #5

Except for a short window around the release of GPT-4 (especially the inflated claims around beating expert trained humans at legal and math tests, as well as "replacing google"), I think people have more or less right-sized their expectations for large language models and generative AI. Clearly it can do interesting and impressive things but it's not superintelligence, and the folks predicting we're just around the…

I agree with what you say above, but my perception is that most people still view the current crop of models as a step or two away from superintelligence. That superintelligence, or AGI, is a matter of continued improvement along the current lines, rather than along entirely different lines.

I like to think of the 'car factory' analogy - it's populated by robots that are in some respects far superior to humans, and are doing 90% of the labor. Some ancient futurist, not having seen one before, could correctly predict that 9 out of 10 jobs will be done by robots, and arrive at the incorrect conclusion that robots have rendered humans obsolete.

In actuality, humans are still needed for the 10% the robots can't do well, or serve to enhance the productivity of humans.

I predict AI is like this and going to be for a while - it can clearly do some stuff well and sometimes better than humans, but humans will have their niches for a while.

Re: Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

#17

Post by [deleted] »

karaterobot wrote: post #9
dekhn wrote: post #5

Except for a short window around the release of GPT-4 (especially the inflated claims around beating expert trained humans at legal and math tests, as well as "replacing google"), I think people have more or less right-sized their expectations for large language models and generative AI. Clearly it can do interesting and impressive things but it's not superintelligence, and the folks predicting we're just around the…

I agree with what you say above, but my perception is that most people still view the current crop of models as a step or two away from superintelligence. That superintelligence, or AGI, is a matter of continued improvement along the current lines, rather than along entirely different lines.

[deleted]

Re: Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

#18

Post by dekhn »

karaterobot wrote: post #9
dekhn wrote: post #5

Except for a short window around the release of GPT-4 (especially the inflated claims around beating expert trained humans at legal and math tests, as well as "replacing google"), I think people have more or less right-sized their expectations for large language models and generative AI. Clearly it can do interesting and impressive things but it's not superintelligence, and the folks predicting we're just around the…

I agree with what you say above, but my perception is that most people still view the current crop of models as a step or two away from superintelligence. That superintelligence, or AGI, is a matter of continued improvement along the current lines, rather than along entirely different lines.

I don't think we can really say what path would lead to superintelligence (for whichever definition you desire) in the near future. Perhaps it is technically possible to achieve merely by making an embodied agent with enough different tricks in a single model (which I see as a matter of continued improvement along current lines), or maybe it requires several new things we haven't conceived yet.

Personally, my area of interest is scientific discovery. Could a model not dissimilar from what we have today, if asked a cogent question, not answer it with an experiment that could be carried out? For example, one of the most important experiments, Avery-MacCleod, which proved (to the extent that you can prove anything in biology) that DNA, not protein, was the primary element of heredity, is not all that complicated, and the mechanical details seem nearly in reach of modern ML techniques. Similarly, could the ML model provide a significant advance in the area of understanding the molecular function in intimate detail of proteins as determined by their structure (which AlphaFold does not do, yet), complete with experimental instructions on how to verify these hypotheses? As of this time, my review of modern ML methods for science suggest we have made some advances, but still have not passed the "phase transition" demonstrating superscientist-level understanding of any field. But perhaps it will just fall out naturally from improved methods for media generation/parsing and ad targeting.

I continue to remain hopeful that within my remaining 20-40 or so years (I'm a typical american male, age 51, with a genome that contains no known risk factors) I will see something like what Vinge describes in https://edoras.sdsu.edu/~vinge/misc/singularity.html in a way that is demonstrable and safe, but honestly, I think it could go in any number of directions from "grim meat-hook future" to "unexpected asteroid takes out human life on the planet, leaving tardigrades to inherit the earth" to "kardyshev-scale civilization".

Re: Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

#19

Post by sailingparrot »

dekhn wrote: post #5

Except for a short window around the release of GPT-4 (especially the inflated claims around beating expert trained humans at legal and math tests, as well as "replacing google"), I think people have more or less right-sized their expectations for large language models and generative AI. Clearly it can do interesting and impressive things but it's not superintelligence, and the folks predicting we're just around the…

> and the folks predicting we're just around the corner have been recognized once again as shysters, hucksters, and charlatans

Why? Can you see the future? No one (serious) was claiming that GPT-4 is superintelligence, it’s about the rate of improvement.

There has only been 6 years between GPT-1 and GPT-4, and each iteration brought more and more crazy emergent behaviour. We still don’t see any sign of the scaling laws slowing down.

I work in ML research, and personally don’t believe ASI is just around, but I talk everyday to researcher that believe so, they don’t say that to swindle anyone’s money (they have extremely well paid 9 to 5 jobs at Goog/MS/OAI, they aren’t trying to raise money from VCs), they only believe so due to the rate of improvement.

Claiming, barely 18 months after GPT-4, when we haven’t yet seen any result from the next jump in scale, that it’s all baloney is a bit premature.

Btw in research time, 10 years from now is « around the corner ».

Now for the VC-chasing folks, their motivation is an entirely different story.

Re: Goldman on Generative AI: doesn't justify costs or solve complex problems [pdf]

#20

Post by dekhn »

torginus wrote: post #16
karaterobot wrote: post #9

Earlier quoted context omitted.

I agree with what you say above, but my perception is that most people still view the current crop of models as a step or two away from superintelligence. That superintelligence, or AGI, is a matter of continued improvement along the current lines, rather than along entirely different lines.

I like to think of the 'car factory' analogy - it's populated by robots that are in some respects far superior to humans, and are doing 90% of the labor. Some ancient futurist, not having seen one before, could correctly predict that 9 out of 10 jobs will be done by robots, and arrive at the incorrect conclusion that robots have rendered humans obsolete. In actuality, humans are still needed for the 10% the robots ca…

I call this the "filter changing problem". No matter how complex you make the technology, somebody still has to change the oil filter (or do whatever other maintainence is required to keep the system running). Sort of like ML-SRE, for those who are familiar with the concept.
Post reply on HN