Earlier quoted context omitted.
It is Llama-2-70b-chat. I quantized it to 2q_k using `quantize` with llama.cpp.
So your experience isn’t representative of the work presented by this post? Or does llama.cpp use the same technique for quantization?
QuIP#: 2-bit Quantization for LLMs
51–60 of 62 posts
Re: QuIP#: 2-bit Quantization for LLMs
#52I wonder whether quantization to 1-bit would be functional?
Re: QuIP#: 2-bit Quantization for LLMs
#53I wonder whether quantization to 1-bit would be functional?
Re: QuIP#: 2-bit Quantization for LLMs
#54Re: QuIP#: 2-bit Quantization for LLMs
#55I wonder whether quantization to 1-bit would be functional?
Re: QuIP#: 2-bit Quantization for LLMs
#56Earlier quoted context omitted.
"For example, the addition of two numbers, 2+2, can be seen as a repeated application of the multiplication operation," I bet most humans ever born did not know what the word "multiplication" meant. Not sure how your metric is useful.
Sometimes, to achieve some perspective, I also zoom out and declare normal over the estimated +-120,000,000,000 people who have ever lived.
Hint: Perhaps read the original output more carefully: "For example, the addition of two numbers, 2+2, can be seen as a repeated application of the multiplication operation,". Or are you counted in the majority of that 120,000,000,000?
Re: QuIP#: 2-bit Quantization for LLMs
#57Earlier quoted context omitted.
Sometimes, to achieve some perspective, I also zoom out and declare normal over the estimated +-120,000,000,000 people who have ever lived.
I think the point went over your head (which just further proves mine). Hint: Perhaps read the original output more carefully: "For example, the addition of two numbers, 2+2, can be seen as a repeated application of the multiplication operation,". Or are you counted in the majority of that 120,000,000,000?
in the
the spring
Re: QuIP#: 2-bit Quantization for LLMs
#58Re: QuIP#: 2-bit Quantization for LLMs
#59Re: QuIP#: 2-bit Quantization for LLMs
#60Earlier quoted context omitted.
yes. common bit depths to represent pixels range up to 64-bits.
But the physical space?
log2(8,294,400) ≈ 23 (each additional bit doubles the number of unique addresses that can be represented)
so 24 bits are the minimum required to locate a point in 2-d space at the resolution of "4k". when you add in color, 60 bits per pixel.
19 bits for 640x480.