131–140 of 141 posts
Earlier quoted context omitted.
Yeah, memory implies persisted state in the model, this is static lookups separate from the transformer. Still superb, though, there's no reason you can't use other gofai tools vs a static database, to trigger expert systems or formalized reasoning.
It's not gofai. It's locality sensitive hashing published in 2008.
Very interesting. GPT-J is an opensource free alternative to GPT-3 and requires at least 12.1GB memory to run the model (which is reduced from original 48GB ram). But if the model stores some kind of index and does internet searches (or hard drive) instead, then it could scale much further as there is a limit on how much memory you can use in production.
>48GB ram 48GB VRAM? 48+ gigabytes of system ram is cheap, 48 gigabytes of ram on a GPU is still painfully expensive.
Earlier quoted context omitted.
I think the person you're replying to is asking for your specific argument. As in what exactly do you think is the advantage gained by quantum computing and why is it important.
Yeah I have no horse in this race, just curious of your viewpoint as someone who works in the field!
Earlier quoted context omitted.
That proportions in that analogy seem way off. Every single neuron of the human brain could be utilizing quantum mechanical effects at the same time. Not very much at all like a kid's sand castle against the world's tides. I don't think anyone really understand "thinking" and quantum mechanics well enough for the mechanisms to be apparent. I mean, squirrels convert forest detritus into general intelligence. I don't u…
Well, quantum computers are just fast but limited classical computers. So there is nothing actually very interesting to be gained by assuming that the brain could be a quantum computer isntead of a classical computer - at best, it would explain why the brain is more efficient at certain computations, but it can't explain how intelligence works. To be clear, any quantum computation can be simulated on a classical comp…
Very interesting. GPT-J is an opensource free alternative to GPT-3 and requires at least 12.1GB memory to run the model (which is reduced from original 48GB ram). But if the model stores some kind of index and does internet searches (or hard drive) instead, then it could scale much further as there is a limit on how much memory you can use in production.
Just doing some napkin math, the whole GPT-J corpus was around 500 billion tokens, which at 4 tokens per byte would be roundabout 2 Terabyte. That, parked on a fast NVMe SSD will give you roundabout 1MM random lookups per second. Even with some transfers inbetween, this should be more than enough to not just perform in equal time, but probably less — as well as cost you less than the GPU you need for the reduced size…
"Only" 825 GB actually: https://pile.eleuther.ai/
A not-insignificant fraction of that is definitively copyrighted material, though, which raises some interesting questions when switching to a model of distributing "a smaller trained model plus the original raw training data" (though it seems that the team behind GPT-J are clearly happy to distribute their full set of data anyway, and seem to be enough under the radar to not attract the wrong sort of attention,at least for now).
Earlier quoted context omitted.
>48GB ram 48GB VRAM? 48+ gigabytes of system ram is cheap, 48 gigabytes of ram on a GPU is still painfully expensive.
Yes, in this GPU market that's essentially a new car's worth of cash.
Earlier quoted context omitted.
Just doing some napkin math, the whole GPT-J corpus was around 500 billion tokens, which at 4 tokens per byte would be roundabout 2 Terabyte. That, parked on a fast NVMe SSD will give you roundabout 1MM random lookups per second. Even with some transfers inbetween, this should be more than enough to not just perform in equal time, but probably less — as well as cost you less than the GPU you need for the reduced size…
> Just doing some napkin math, the whole GPT-J corpus was around 500 billion tokens, which at 4 tokens per byte would be roundabout 2 Terabyte. "Only" 825 GB actually: https://pile.eleuther.ai/ A not-insignificant fraction of that is definitively copyrighted material, though, which raises some interesting questions when switching to a model of distributing "a smaller trained model plus the original raw training data"…
Seems odd to claim 25x reduction in size when the algo involves looking into a database of a trillion chunks of text.
The word "Algo" here is definitely awkward. The point is though that what matters most here is the number of parameters, as those correlate quite closely with training and inference time. Storage space is pretty trivial, but TPU cycles are less so.
Earlier quoted context omitted.
> Just doing some napkin math, the whole GPT-J corpus was around 500 billion tokens, which at 4 tokens per byte would be roundabout 2 Terabyte. "Only" 825 GB actually: https://pile.eleuther.ai/ A not-insignificant fraction of that is definitively copyrighted material, though, which raises some interesting questions when switching to a model of distributing "a smaller trained model plus the original raw training data"…
Not pointing out such potential problems in public forums is likely to extend the possibility that it remains readily available.