221–230 of 313 posts
Earlier quoted context omitted.
The difference is small, UNTIL you get to 4 bit quantization, where the model is noticeably dumber. 8 bits, imo, is the minimum.
So which is better, running 7B without quantization or running 13B with? They both require about the same amount of vRAM (10gb).
Earlier quoted context omitted.
The difference is small, UNTIL you get to 4 bit quantization, where the model is noticeably dumber. 8 bits, imo, is the minimum.
WRONG. Research shows effectively imperceptible performance difference at 4-bit and even 3-bit with GPTQ quantization. You cannot tell the difference and if you think you do you're wrong, because it barely even registers on any benchmark. (Note: llama.cpp's 4bit is naive, not GPTQ, and sucks but they are refactoring it to use GPTQ quantization) References: https://arxiv.org/abs/2210.17323 - GPTQ: Accurate Post-Traini…
Like, 4 bit quantization really is probably enough for a number of usecases and likely beats smaller models with precision enough to make it the equivalent number of bits, but this really is only presenting half of the story. "You cannot tell the difference and if you think you do you're wrong, because it barely even registers on any benchmark" can be regarded as antagonistic, and also really doesn't line up with reality in a number of usecases. Sure, maybe for some models, UINT4 quantization is good enough. But there's a very large space of model architectures and problems, even for language learning, many of which do have very demonstrable drops in performance. And at certain perplexity levels, every bit (heh) matters.
In any case, an argument for moderation, please.
The training data doesn't seem to be great quality... "instruction": "Calculate the sum of each column in the following table.", "input": "2 | 3 | 4\n 5 | 6 | 7", "output": "2+3+4 = 9\n5+6+7 = 18" I think better results could be obtained by manually going through these 52,000 training examples - even a couple of seconds per example should be enough to weed out obviously wrong ones, and should only take <$1000 of huma…
Training a model to do math is impossible. If you tell a child that 1+1=2 10+10=20 100+100=200 an "AI" can not figure out that 1000+1000=2000 like a child can.
Earlier quoted context omitted.
For 10 billion+ parameter models, the effects of quantization are relatively small, for smaller models like Llama 7B the effect becomes more dramatic, but there is ongoing research on new quantization methods (like GPTQ) that preserve significant performance even on the lower end. Quantization isn't the only technique available for downsizing a model, Llama itself is already the result of sizing the model and input d…
Curious about the google traffic comment. Are you saying people are visiting sites less because they can stay on Bing/OpenAI?
I am surprised that they were able to launch this in stanford's domain. They clearly broke TOS of both Facebook and OpenAI, and even admitted doing that. I would be happy if the research decides to ignore openAI and facebook's useless restrictions.
The training data doesn't seem to be great quality... "instruction": "Calculate the sum of each column in the following table.", "input": "2 | 3 | 4\n 5 | 6 | 7", "output": "2+3+4 = 9\n5+6+7 = 18" I think better results could be obtained by manually going through these 52,000 training examples - even a couple of seconds per example should be enough to weed out obviously wrong ones, and should only take <$1000 of huma…
Don’t even need human time. Just run it through davinci. Is this correct, yes or no? Or MTurk. Can cost less than $500.
I wonder how davinci would respond if every single output was fed back into itself with a prompt like "does this contain any errors?". That probably wouldn't work very well.
Do we have some studies on the impact of extreme quantization on these models? In my understanding, the efforts to make them run in commodity hardware involves the quantization of the weights to 8 bits or even 4 bits. Is the model still reliable after this?
Yes, intro on the topic if anyone is interested, include topic of benchmarking degradation: - https://huggingface.co/blog/hf-bitsandbytes-integration
Earlier quoted context omitted.
ChatGPT is fine-tuned InstructGPT, which has 1.3B parameters, if I'm not mistaken. Reference for the former: https://www.technologyreview.com/2023/03/03/1069311/inside-s...
InstructGPT isn't a single model, it's a set of techniques for finetuning a foundation model
Does it mean we load 175B gpt-3 model first, then overwrite 1.3B parameters with InstructGPT?
I find this sentence difficult to understand
> Our labelers prefer outputs from our 1.3B InstructGPT model over outputs from a 175B GPT-3 model
https://openai.com/research/instruction-following
I am a newbie, plz correct me if I am wrong.
Earlier quoted context omitted.
LLM just predicts the next token given the previous tokens(this can be trained without manual labelling by humans). Instruct GPT and ChatGPT use reinforcement learning from human feedback to align the model with human intents so it understands instructions. https://huggingface.co/blog/rlhf
Note that Alpaca is NOT using RLHF. It explicitly states it used supervised finetuning.
> We train the Alpaca model on 52K instruction-following demonstrations generated in the style of self-instruct using text-davinci-003
Which leads to self-instruct https://github.com/yizhongw/self-instruct
From a glimpse they used a LM to classify instructions & train the model which IMHO is very similar to RLHF