Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sparks of Artificial General Intelligence: Early Experiments with GPT-4

arxiv.org

Post reply on HN
Print view View original

191–200 of 244 posts

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#191

Post by gwern »

nopinsight wrote: post #52
outlace wrote: post #14

ChatGPT and its relatives are very very impressive on first impressions, but I've been using ChatGPT-3 and now 4 heavily every day since they became available to individuals and once you start using them this much it becomes very clear how NOT intelligent they are. It really just seems like extremely impressive statistical inference after this much use and finding so many failure modes. But it is still impressive how…

It depends on 1) the domains 2) your comparison group. On 2), many software engineers and computer scientists compare these language models' logic and creative problem solving abilities with themselves and their peer group. But they are usually 1-2+ SD above average humans at these things. (Note: Someone gave GPT-4 an IQ test and the result was 96, slightly below the average of reference human group at 100. The SD of…

Those rhyme riddles are pretty impressive. It may not truly understand rhymes due to BPEs, but I guess it can go a long way with an immense vocab, perfect recall, and memorization of similar-sounding words to beat ordinary human players who aren't scoring 800 SAT-Vs...

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#192

Post by shmed »

bumbledraven wrote: post #63

GPT-4's proof of the infinitude of primes, as quoted in the paper, is wrong. Yes, I think I can, Though it might take a clever plan. I’ll start by noting Euclid’s proof, Which shows that primes aren’t just aloof. Assume we have a finite list, Of primes, and that none have been missed. Multiply them all together, And add one, just to be clever. The result will be a number, That has no prime factors, oh wonder! Not tru…

You literally just proved it right

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#193

Post by apidercondo »

Mike_12345 wrote: post #59
letitgo12345 wrote: post #57

Earlier quoted context omitted.

Where are you getting the 200 number from?

I must have hallucinated that. GPT-3 has 96 layers but they haven't disclosed the number of layers in GPT-4.

Interesting how we are already starting to use the lingo in the rest of our lives.

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#194

Post by geophile »

killerstorm wrote: post #183
geophile wrote: post #35

Earlier quoted context omitted.

> Surely these scientists ... aren't so stupid as to think emergent behavior potentially resembling intelligence could result from such simple systems? It's just statistics after all. Why is that a stupid thought? What is so preposterous about "just statistics" -- with billions of nodes, and extensively trained, producing intelligent behavior? The implicit assumption is that human brains are doing something else, or…

> Why is that a stupid thought? What is so preposterous about "just statistics" Suppose you have N variables x_1, ..., x_10 and you want to predict y_1, ..., y_10. You know that each y_i depend on each x_i in a complex, non-linear way. How many samples would you need to to make sense of distribution? How does number of samples grow with N?

I have no idea what that has to do with the points you are responding to.

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#195

Post by HopenHeyHi »

voisin wrote: post #161
HopenHeyHi wrote: post #80

Earlier quoted context omitted.

6. Meta comment for karma whoring 7. Like 6, but funnier A. Joke thread pile on B. Reprimands from humorless C. Dejected mods having to clean it all up

> B. Reprimands from humorless You nailed it with this. I’ve never given it enough attention but this is totally bang on, found is a huge portion of highly commented posts.

These things ebb and flow my dear Adso: https://www.youtube.com/watch?v=jUUB96c6EpY

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#196

Post by js8 »

cjbprime wrote: post #154
js8 wrote: post #114

Earlier quoted context omitted.

Feynman had a really nice story about how he was into puzzles when he was at Princeton. It took him a while to solve the new ones, but eventually he learned all the well-known instances so he could answer instantly. It made him a genius in other people's eyes. All I want from AGI is to demonstrate that it can solve a straightforward logic problems (puzzles, if you will), that it provably didn't see before. Or at leas…

There's a 150-page paper solely to describe instances of it doing that. It's the article attached to this comment thread.

Well, it's about the standard of the proof. When I say "demonstrate", I don't mean just experimentally, I mean theoretically, to show that the algorithm is capable of reasoning about potentially arbitrarily large instances of puzzles.

That's what the experiments have shown - once the unknown instance gets large enough, the reasoning of LLM breaks down. This is not the case with humans, who can, as noted elsewhere, do a tree search, form hypotheses, etc.

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#197

Post by carapace »

Robotbeat wrote: post #176
carapace wrote: post #19

Earlier quoted context omitted.

It seems clear to me that these systems think in a meaningful sense, but I don't think they are beings . In Cybernetics there is a result that says that any well-regulated system must contain a model of itself. This seems as good a definition as any of a "being", and by this definition these language models don't make the cut.

The architecture for large language models is summarized in the training set for large language models. With fairly minimal modification such as via a plug-in, chatGPT and the like are Turing complete and can thus model themselves.

Hmm, then, from first principles, we should expect "ghosts" to arise in those systems. (These will not be the "virtual entities" that people talk to and call by name (Alexa, Cortana, Siri, etc., but more akin to fixed points in the flow of information.)

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#198

Post by coldtea »

neilellis wrote: post #169

[This is in reply to the comments not the article!] It's just a statistical model is the logical equivalent of human beings are just a bunch of atoms. The amount of reductionist thinking that goes on in tech is hilarious. First define AGI then challenge an AI to meet those requirements. If it meets them it is AGI. Put aside your preconceptions of what technology you think is required to achieve the goals and stay emp…

>It's just a statistical model is the logical equivalent of human beings are just a bunch of atoms.

Not exactly.

One says "human beings are just a bunch of atoms" referring to the low level constituans (in a reductionistic way), but not making an accessment about the abilities emerging from those atoms in their interactions when in the form of a human.

But when one says that GPT is "just a statistical model" they're implying a capacity cap of statistical models, that makes modelling certain thinking behavior impossible (regarless of how impressive the current results are, they might very well be capped to go beyond some limit because of the method -statistically model- involved).

So, you can consider "GPT is just a statistical model" analogous to:

"This engine can't parse a context senstive language because it's just a regular expression engine".

>First define AGI then challenge an AI to meet those requirements. If it meets them it is AGI. Put aside your preconceptions of what technology you think is required to achieve the goals and stay empirical.

The problem is definitions can be slippery, and even famous tests (like the Turing Test) might be found lacking in practice, as we discover that, yes, it can pass this test, but there's still ways off what we consider human-like performance in many areas. So, we should also stay empirical about the definitions, tests, and goals too.

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#199

Post by nl »

golol wrote: post #178
Mike_12345 wrote: post #36

Earlier quoted context omitted.

GPT-4 is often overhyped and underhyped because few really understand it. It's not a Markov Chain or a fancy text predictor. It's a ~200 layer neural network that models a vast hierarchy of concepts through language. It has emergent properties that we don't yet understand.

it is a markov chain; At least the underlying decoder only transformer is.

It's not. It can do in context learning, which Markov chains cannot do.

Re: Sparks of Artificial General Intelligence: Early Experiments with GPT-4

#200

Post by coldtea »

_gabe_ wrote: post #13

> Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. But it's just statistics, a fancy text predictor, a Markov-chain. Surely these scientists that work in the field of AI and are intimately familiar with how this stuff works aren't so stupid as to think emergent behavior pote…

>But it's just statistics, a fancy text predictor, a Markov-chain. Surely these scientists that work in the field of AI and are intimately familiar with how this stuff works aren't so stupid as to think emergent behavior potentially resembling intelligence could result from such simple systems?

Well, it has already shown "emergent behavior potentially resembling intelligence", like answering questions and performing complex tasks, so there's that.

You might argue "but it makes mistakes", but people, even very intelligent ones also do make mistakes.

You might also argue "but it's just text and statistics". Well, snd a computer is just very simple logical gates doing very simple operations. It can be done even entirely with NAND gates. Still most scientists do believe that a computer can model human intelligence given a model of the brain to run.

So if it can do what a human does by just using very simple interactions from very simple NAND gates, why would statistical processing, which can be even more elaborate, fare worse? Heck, given the appropriate training input it might even be feasible to build a turing machine inside the weighted LLM.

You might also argue "but its intelligence is just based on its training set". Well, how would a human perform without their own training set? Memories, education, sensory input, feedback mechanisms like pain and touch, and so on?

>It's just looking up the answers. No hint of intelligence. Just a mindless machine.

This is just taking its own premise for granted. If anything, this argument shows "no hint of intelligence".

Post reply on HN