Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ternary Bonsai: Top Intelligence at 1.58 Bits

prismml.com

Post reply on HN
Print view View original

21–30 of 60 posts

Re: Ternary Bonsai: Top Intelligence at 1.58 Bits

#21

Post by AlotOfReading »

Animats wrote: post #7
yodon wrote: post #4

So excited to see this - the big advantage of 1.58 bits is there are no multiplications at inference time, so you can run them on radically simpler and cheaper hardware.

At 4 bits, you could just have a hard-wired table lookup. Two 4 bit values in, 256 entry table. You can have saturating arithmetic and a post-processing function for free. Somebody must be building hardware like that.

A LUT is pretty wasteful. You only have a one bit significand, so the mantissa and sign bits are boolean binops, and the exponent is a 2 bit adder.

Re: Ternary Bonsai: Top Intelligence at 1.58 Bits

#22

Post by sbierwagen »

freakynit wrote: post #18
usernametaken29 wrote: post #17

I think it’s exciting to live in this quirky universe where we have simply accepted our hardware does weird and nonlinear stuff and that powers some math and that’s why your transform function works. Many people thought quantisation is not viable to the extent we see, but we clearly underestimated the effect of hardware on the actual non linearity of the models. Cool to see this pushed to the limits.

Nature has already set an absurdly high bar. The human brain runs on roughly 20 watts, yet delivers a level of intelligence we still can't clearly define, let alone replicate. Nothing we've built comes close... either in capability or efficiency. We're still very early in understanding what "intelligence" even means, much less engineering it. so, we have a long way to go, and push.

Depending on how you convert synapse count to parameters, the brain also has something like a thousand trillion parameters. In that light it's pretty darn surprising that an artificial neural network can produce anything like coherent text.

Re: Ternary Bonsai: Top Intelligence at 1.58 Bits

#23

Post by eru »

freakynit wrote: post #18
usernametaken29 wrote: post #17

I think it’s exciting to live in this quirky universe where we have simply accepted our hardware does weird and nonlinear stuff and that powers some math and that’s why your transform function works. Many people thought quantisation is not viable to the extent we see, but we clearly underestimated the effect of hardware on the actual non linearity of the models. Cool to see this pushed to the limits.

Nature has already set an absurdly high bar. The human brain runs on roughly 20 watts, yet delivers a level of intelligence we still can't clearly define, let alone replicate. Nothing we've built comes close... either in capability or efficiency. We're still very early in understanding what "intelligence" even means, much less engineering it. so, we have a long way to go, and push.

> Nothing we've built comes close... either in capability or efficiency.

Only when you look at stuff that the brain is specifically good at.

You can surpass the brain with even simple mechanical adders or an abacus in certain subdomains.

Re: Ternary Bonsai: Top Intelligence at 1.58 Bits

#24

Post by freakynit »

eru wrote: post #23
freakynit wrote: post #18

Earlier quoted context omitted.

Nature has already set an absurdly high bar. The human brain runs on roughly 20 watts, yet delivers a level of intelligence we still can't clearly define, let alone replicate. Nothing we've built comes close... either in capability or efficiency. We're still very early in understanding what "intelligence" even means, much less engineering it. so, we have a long way to go, and push.

> Nothing we've built comes close... either in capability or efficiency. Only when you look at stuff that the brain is specifically good at. You can surpass the brain with even simple mechanical adders or an abacus in certain subdomains.

General intelligence I mean. What calculations even need to be performed and when, still comes from our brains.

Re: Ternary Bonsai: Top Intelligence at 1.58 Bits

#25

Post by freakynit »

sbierwagen wrote: post #22
freakynit wrote: post #18

Earlier quoted context omitted.

Nature has already set an absurdly high bar. The human brain runs on roughly 20 watts, yet delivers a level of intelligence we still can't clearly define, let alone replicate. Nothing we've built comes close... either in capability or efficiency. We're still very early in understanding what "intelligence" even means, much less engineering it. so, we have a long way to go, and push.

Depending on how you convert synapse count to parameters, the brain also has something like a thousand trillion parameters. In that light it's pretty darn surprising that an artificial neural network can produce anything like coherent text.

It indeed is. We now have models less than 100M params producing pretty coherent, and somewhat relevant text to give input. That is indeed impressive.

I believe the answer lies in how "quickly" (and how?) we are able to learn, and then generalize those learnings as well. As of now, these models need millions (at least) examples to learn, and are still not capable of generalizing the learnings to other domains. Human brains hardly need a few, and then, they generalize those pretty well.

Re: Ternary Bonsai: Top Intelligence at 1.58 Bits

#29

Post by zkmon »

The raw math: File size is hard-linked to parameter count and quant type. Intelligence is sort of linked to parameter count. Parameter count dictates the hardware requirement. What't left for the labs is, compressing more intelligence into lower parameter count, or packing more of specialized intelligence or buying up more hardware. Those are the only 3 directions all models/labs are heading.

Re: Ternary Bonsai: Top Intelligence at 1.58 Bits

#30

Post by sally_glance »

sbierwagen wrote: post #22
freakynit wrote: post #18

Earlier quoted context omitted.

Nature has already set an absurdly high bar. The human brain runs on roughly 20 watts, yet delivers a level of intelligence we still can't clearly define, let alone replicate. Nothing we've built comes close... either in capability or efficiency. We're still very early in understanding what "intelligence" even means, much less engineering it. so, we have a long way to go, and push.

Depending on how you convert synapse count to parameters, the brain also has something like a thousand trillion parameters. In that light it's pretty darn surprising that an artificial neural network can produce anything like coherent text.

Maybe the brain is more akin to a network of networks and the actual reasoning part is not all that large? There are lots of areas dedicated exclusively to processing input and controlling subsystems. I can imagine a future where large artificial networks work in a similar way, with multiple smaller ones connected to each other.
Post reply on HN