Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I asked GPT-NeoX-20B a hundred arithmetic questions

twitter.com

Post reply on HN
Print view View original

121–130 of 235 posts

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#121

Post by guantanamo_bob »

tehsauce wrote: post #84

The latest version of gpt-3 spooked me the other day with: Me: Can here you here still here understand here what here I here am here asking here if here I here put here words here in here between? Gpt: Yes, I can understand what you are asking if you put words in between. Me: What was the word that I put in between? Gpt: The word that you put in between is "here."

Must :clap: have :clap: learned :clap: that :clap: from :clap: all :clap: the :clap: clapping :clap: that :clap: goes :clap: on :clap:.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#122

Post by rurp »

defaultprimate wrote: post #108
jquery wrote: post #106

Earlier quoted context omitted.

Imho, not a matter of "if" but "when". I'm convinced that it will be a future civil rights battle, with young people largely on the "AI has rights" side and old people largely on the "AI has no rights" side.

Then you grossly misunderstand how far along AI is. AGI is not even a remote possibility with current techniques and implementations (and I would contest, entirely impossible with digital logic). It's just massive amount of statistics that were computationally impossible given available hardware until recently. We don't have a baseline understanding of consciousness or intuition to a degree that we could even begin t…

While true, what you're saying is totally tangential to whether or not large numbers of people will treat AIs like they're conscious.

Expecting masses of people to defer to subject matter experts, contrary to what their feelings tell them, isn't a bet I would have much confidence in given the current climate.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#123

Post by progforlyfe »

defaultprimate wrote: post #108
jquery wrote: post #106

Earlier quoted context omitted.

Imho, not a matter of "if" but "when". I'm convinced that it will be a future civil rights battle, with young people largely on the "AI has rights" side and old people largely on the "AI has no rights" side.

Then you grossly misunderstand how far along AI is. AGI is not even a remote possibility with current techniques and implementations (and I would contest, entirely impossible with digital logic). It's just massive amount of statistics that were computationally impossible given available hardware until recently. We don't have a baseline understanding of consciousness or intuition to a degree that we could even begin t…

I agree with that sentiment. Also I would estimate that AI would eliminate humans long before it would or could reach the level of what humans are. So we wouldn't exist to see such a world.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#124

Post by newsbinator »

defaultprimate wrote: post #108
jquery wrote: post #106

Earlier quoted context omitted.

Imho, not a matter of "if" but "when". I'm convinced that it will be a future civil rights battle, with young people largely on the "AI has rights" side and old people largely on the "AI has no rights" side.

Then you grossly misunderstand how far along AI is. AGI is not even a remote possibility with current techniques and implementations (and I would contest, entirely impossible with digital logic). It's just massive amount of statistics that were computationally impossible given available hardware until recently. We don't have a baseline understanding of consciousness or intuition to a degree that we could even begin t…

If you want to get to AGI, it doesn't really have to be a remote possibility with current techniques.

The only thing current techniques would have to do is build something that improves itself intelligently enough.

Then the next iteration and the next, until you get to an iteration in which AGI is a remote possibility with current techniques and implementations.

We're currently far from AGI, but I'm not sure we're far from a thing that can make a thing that makes a thing that makes something like AGI.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#125

Post by rexreed »

Amazing how a very costly to train system using billions of neural nodes on millions of dollars of compute performs more poorly than an 8-bit 1970s pocket calculator.

Not sure why people are expecting some sort of "intelligence" to emerge from a text generator model trained on Internet corpus data. GPT-3 doesn't calculate, it pattern matches.

I do get why people might be surprised, on the other hand, that it actually doesn't perform worse than indicated here. Maybe it's surprising upside. But since we know that the GPT is a transformer model, what it is doing is applying a probabilistic best-fit. From this perspective I can see how it is best-fitting data in ways that can provide these sorts of results, especially given all that training data.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#126

Post by kripke »

ggrrhh_ta wrote: post #62
sillysaurusx wrote: post #51

Earlier quoted context omitted.

Try temp 0.1 top-k 40. For math, it matters to have an unthinkably low temperature. It’s what generated the results in the OP. What is 12345 - 12345? 0 What is the distance between -0.1 and -0.01? 0.09 One neat example just now: What is 12345 divided by 12? 4115/4

I pointed it out above; even though it is text, the ASCII representation is just a different base for the numbers - base 2^8 - ('325' is '3' * (2^32) + '2' * (2^16) + '5' * 2^8 = 51 * 2^32 + 50 * 2^16 + 53 * 2^8); it should approximate those polynomial functions very well.

Does GPT know about ASCII? My understanding was that these models use a dictionary of (initially) random vectors as input and learn their own text representation.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#127

Post by andai »

marcodiego wrote: post #103
supermdguy wrote: post #96

Earlier quoted context omitted.

It's interesting, in the forums for the beta program there have been already been a few people making posts where they're convinced that the AI is conscious. That's never really been something I've thought about much since I know a little about how it works, but I could totally see how someone who didn't have as much context for how GPT-3 works could see it as some sort of sentience. https://community.openai.com/t/a-…

Long time ago I wrote a small program. It simulated a simple world with creatures and food. Creatures had "energy" which was lost when they moved, when energy was low, they "looked for food close by" to feed. When energy was high, they "looked for a suitable partner close by" to reproduce. When they reached food, they gained energy, when they reached a "suitable partner" they turned in 3 creatures with combined energ…

I guess you didn't implement save/load? :)

That's super cool, did you ever put it online?

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#128

Post by andai »

newsbinator wrote: post #124
defaultprimate wrote: post #108

Earlier quoted context omitted.

Then you grossly misunderstand how far along AI is. AGI is not even a remote possibility with current techniques and implementations (and I would contest, entirely impossible with digital logic). It's just massive amount of statistics that were computationally impossible given available hardware until recently. We don't have a baseline understanding of consciousness or intuition to a degree that we could even begin t…

If you want to get to AGI, it doesn't really have to be a remote possibility with current techniques. The only thing current techniques would have to do is build something that improves itself intelligently enough. Then the next iteration and the next, until you get to an iteration in which AGI is a remote possibility with current techniques and implementations. We're currently far from AGI, but I'm not sure we're fa…

I still think we'd be better off making a thing that makes smarter humans.

Re: I asked GPT-NeoX-20B a hundred arithmetic questions

#129

Post by marcodiego »

andai wrote: post #127
marcodiego wrote: post #103

Earlier quoted context omitted.

Long time ago I wrote a small program. It simulated a simple world with creatures and food. Creatures had "energy" which was lost when they moved, when energy was low, they "looked for food close by" to feed. When energy was high, they "looked for a suitable partner close by" to reproduce. When they reached food, they gained energy, when they reached a "suitable partner" they turned in 3 creatures with combined energ…

I guess you didn't implement save/load? :) That's super cool, did you ever put it online?

Didn't implement save. It was the late 90's and eventually lost the HD. Never put it online. Don't judge me, I did in Delphi (C++ Builder, actually). Simple and pleasurable to use IDE. I've been thinking about re-doing it in processing or p5, but never cared enough. It wouldn't be too much work though. Certainly a good exercise to learn a new language.
Post reply on HN