[deleted]
DeepMind’s new AI with a memory outperforms algorithms 25 times its size
11–20 of 141 posts
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#12A neural net with access to wikipedia is faster than than a neural net that contains Wikipedia? Seems odd to call it AI with a memory though... unless I'm misunderstanding. It's more like AI with a decent memory and an understanding of how to use an encyclopedia.
Yeah, memory implies persisted state in the model, this is static lookups separate from the transformer. Still superb, though, there's no reason you can't use other gofai tools vs a static database, to trigger expert systems or formalized reasoning.
I got into a very long debate with an openai person 4/5 years ago about this + adversarial learning + access to a quantum computer (think just straight up world class abacus) was close to the primitives required for more generalized AI. They didn't agree with me, but that's ok! :)
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#13Earlier quoted context omitted.
Yeah, memory implies persisted state in the model, this is static lookups separate from the transformer. Still superb, though, there's no reason you can't use other gofai tools vs a static database, to trigger expert systems or formalized reasoning.
I got into a very long debate with an openai person 4/5 years ago about this + adversarial learning + access to a quantum computer (think just straight up world class abacus) was close to the primitives required for more generalized AI. They didn't agree with me, but that's ok! :)
What’s your argument?
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#14Seems odd to claim 25x reduction in size when the algo involves looking into a database of a trillion chunks of text.
The word "Algo" here is definitely awkward. The point is though that what matters most here is the number of parameters, as those correlate quite closely with training and inference time. Storage space is pretty trivial, but TPU cycles are less so.
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#15Very interesting. GPT-J is an opensource free alternative to GPT-3 and requires at least 12.1GB memory to run the model (which is reduced from original 48GB ram). But if the model stores some kind of index and does internet searches (or hard drive) instead, then it could scale much further as there is a limit on how much memory you can use in production.
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#16If we point it at the horrendously bad scots wiki (some kid in the US decided he'd translate Wikipedia into what he thought was lowland scots/Doric.. it's a disaster) we might get entertainingly bad outcomes.
Note, the stuff written by said kid has long since been deleted. However i have no idea what the quality of the rest of scowiki is.
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#17Earlier quoted context omitted.
Yeah, memory implies persisted state in the model, this is static lookups separate from the transformer. Still superb, though, there's no reason you can't use other gofai tools vs a static database, to trigger expert systems or formalized reasoning.
I got into a very long debate with an openai person 4/5 years ago about this + adversarial learning + access to a quantum computer (think just straight up world class abacus) was close to the primitives required for more generalized AI. They didn't agree with me, but that's ok! :)
> + access to a quantum computer
Yeah... i think i already agree with the person disagreeing with you just based on this description. "world class abacus" is not an accurate description of what a quantum computer is.
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#18Earlier quoted context omitted.
Yeah, memory implies persisted state in the model, this is static lookups separate from the transformer. Still superb, though, there's no reason you can't use other gofai tools vs a static database, to trigger expert systems or formalized reasoning.
I got into a very long debate with an openai person 4/5 years ago about this + adversarial learning + access to a quantum computer (think just straight up world class abacus) was close to the primitives required for more generalized AI. They didn't agree with me, but that's ok! :)
That poor researcher.
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#19Earlier quoted context omitted.
Yeah, memory implies persisted state in the model, this is static lookups separate from the transformer. Still superb, though, there's no reason you can't use other gofai tools vs a static database, to trigger expert systems or formalized reasoning.
I got into a very long debate with an openai person 4/5 years ago about this + adversarial learning + access to a quantum computer (think just straight up world class abacus) was close to the primitives required for more generalized AI. They didn't agree with me, but that's ok! :)
Does anyone know if we understand enough about natural, generally intelligent brains to dismiss the idea that they are using quantum phenomenon for computation? Is it unlikely for any reason?
Re: DeepMind’s new AI with a memory outperforms algorithms 25 times its size
#20If we point it at the horrendously bad scots wiki (some kid in the US decided he'd translate Wikipedia into what he thought was lowland scots/Doric.. it's a disaster) we might get entertainingly bad outcomes.
Note, the stuff written by said kid has long since been deleted. However i have no idea what the quality of the rest of scowiki is.
It's not awful, but I feel it's still pretty meh. I say this as a person raised in Edinburgh in the sixties and seventies. How bad? Well.. in their backend meta pages they link to the DSL (Dictionary of the scots language/dictionars o' Scots Leid [0]) which says this:
Written Scots In the written mode, Scots spelling remains variable. Attempts to make it more consistent, notably the Scots Style Sheet produced by the Makars’ Club in 1947 or the Recommendations for Writers in Scots published by the Scots Language Society in 1985, have had at best only limited success, competing with other systems that have been developed to represent more closely localized varieties of spoken Scots.
When your reference text says the language isn't yet well captured in a single print, you better believe the wiki page is a hot mess.