Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Era of 1-bit LLMs: ternary parameters for cost-effective computing

arxiv.org

Post reply on HN
Print view View original

201–210 of 462 posts

Re: The Era of 1-bit LLMs: ternary parameters for cost-effective computing

#201

Post by nighthawk454 »

gojomo wrote: post #177

That's not a 'bit' ("Binary digIT"). It's closer to a 'trit' ("TeRnary-digIT"). Specifically, ternary digits spanning {-1, 0, 1} (rather than the usual {0, 1, 2} in a base-3 numbering system) are 'balanced ternary'. A great intro to the theoretical reasons ternary might have some promise in computing is this 2001 article from 'American Scientist', "Third Base", which quotes Knuth calling balanced-ternary "perhaps the…

Base e is the optimal base for number representation, so that’s probably why. Followed by base 3, then base 2.

https://en.m.wikipedia.org/wiki/Radix_economy

Re: The Era of 1-bit LLMs: ternary parameters for cost-effective computing

#202

Post by sandyarmstrong »

m3kw9 wrote: post #182
nutanc wrote: post #122

Earlier quoted context omitted.

We have been experimenting with the paper( https://www.researchgate.net/publication/372834606_ON_NON-IT... ). There is a mathematical proof that binary representation is enough to capture the latent space. And in fact we don't even need to do "training" to get that representation. The practical application we tried out for this algorithm was to create an alternate space for mpnet embeddings of Wikipedia paragraphs. U…

How is this not lossy compression?

LLMs and vector embeddings are always lossy compression, yes?

Re: The Era of 1-bit LLMs: ternary parameters for cost-effective computing

#205

Post by dekhn »

gojomo wrote: post #177

That's not a 'bit' ("Binary digIT"). It's closer to a 'trit' ("TeRnary-digIT"). Specifically, ternary digits spanning {-1, 0, 1} (rather than the usual {0, 1, 2} in a base-3 numbering system) are 'balanced ternary'. A great intro to the theoretical reasons ternary might have some promise in computing is this 2001 article from 'American Scientist', "Third Base", which quotes Knuth calling balanced-ternary "perhaps the…

How useful are -0 and 0? You could splurge on two bits per value which gives you { -1, -0, 0, 1 }

Re: The Era of 1-bit LLMs: ternary parameters for cost-effective computing

#206

Post by Razengan »

gojomo wrote: post #177

That's not a 'bit' ("Binary digIT"). It's closer to a 'trit' ("TeRnary-digIT"). Specifically, ternary digits spanning {-1, 0, 1} (rather than the usual {0, 1, 2} in a base-3 numbering system) are 'balanced ternary'. A great intro to the theoretical reasons ternary might have some promise in computing is this 2001 article from 'American Scientist', "Third Base", which quotes Knuth calling balanced-ternary "perhaps the…

Why not a tit?

Re: The Era of 1-bit LLMs: ternary parameters for cost-effective computing

#207

Post by oxxoxoxooo »

Prior art:

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1

https://arxiv.org/abs/1602.02830

Ternary Neural Networks for Resource-Efficient AI Applications

https://arxiv.org/abs/1609.00222

Re: The Era of 1-bit LLMs: ternary parameters for cost-effective computing

#208

Post by baq »

m3kw9 wrote: post #182
nutanc wrote: post #122

Earlier quoted context omitted.

We have been experimenting with the paper( https://www.researchgate.net/publication/372834606_ON_NON-IT... ). There is a mathematical proof that binary representation is enough to capture the latent space. And in fact we don't even need to do "training" to get that representation. The practical application we tried out for this algorithm was to create an alternate space for mpnet embeddings of Wikipedia paragraphs. U…

How is this not lossy compression?

kind of related: https://medium.com/@heinrichpeters/commentary-gzip-knn-beats...

Re: The Era of 1-bit LLMs: ternary parameters for cost-effective computing

#209

Post by emXdem »

w-m wrote: post #133
rfoo wrote: post #123

Earlier quoted context omitted.

Interesting, how do you use -0 in the add, then? Is -0+1-1 a 0 or a -0? > Could the additional -0 carry some pseudo-gradient information It looks like training was done on fp32 or bf16. Low-bit quantization is approximated with STE during training. I'd expect training itself cause each point to "polarize" towards 1 or -1. > 2-bit quantizations being proposed Symmetric (i.e. without 0) exponential values were pretty p…

> how do you use -0 in the add In my mind the two zero values would represent a tiny epsilon around 0, let's say -0.01 and +0.01. Looking at them like this, it would mean +0 +0 -0 = +0 +0 -0 -0 = -0 +1 * +0 = +0 -1 * +0 = -0 Performing addition with the same sign count in each group would be problematic. How to decide on the sign of +0-0 or +1-1, other than flipping a coin?

[dead]

Re: The Era of 1-bit LLMs: ternary parameters for cost-effective computing

#210

Post by jdiff »

Razengan wrote: post #206
gojomo wrote: post #177

That's not a 'bit' ("Binary digIT"). It's closer to a 'trit' ("TeRnary-digIT"). Specifically, ternary digits spanning {-1, 0, 1} (rather than the usual {0, 1, 2} in a base-3 numbering system) are 'balanced ternary'. A great intro to the theoretical reasons ternary might have some promise in computing is this 2001 article from 'American Scientist', "Third Base", which quotes Knuth calling balanced-ternary "perhaps the…

Why not a tit?

Because bi- is two, tri- is three. Ti- is meaningless, and not good enough of a joke to make up for it.
Post reply on HN