51–60 of 314 posts
Having spent quite a bit of time playing around with llama.cpp, alpaca.cpp, loras, and the many other llama-based weights lately, here is my impression: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far. [0] If yo…
Earlier quoted context omitted.
what is it for chatgpt?
4k for gpt-3.5 and 8k-32k for gpt4 (there are multiple versions: https://platform.openai.com/docs/models/gpt-4 )
Is it possible they are doing summarization when you use the web interface to gpt4?
Because I have some extremely long conversations and I ask it if it remembers the beginning of the conversation and it does.
Earlier quoted context omitted.
It already is, as you can have wildly different outcomes for models of the same size, and if you can get a similar output from a ten times smaller model, that's not equivalent, that's overwhelmingly better since it takes ten times less compute and for current model sizes the cost of inference is quite relevant.
We need to see something like "dropout" gain popularity again where you take a trained network and keep removing hidden neurons and re-training a few iterations until you can't obtain the same accuracy anymore, at which point you've reached what should be the smallest that network can be while still encoding the same knowledge. With models of these sizes though, few have the compute or the time to do stuff like that.…
https://blog.dataiku.com/making-neural-networks-smaller-for-... https://arxiv.org/abs/2301.00774 https://tivadardanka.com/blog/how-to-compress-a-neural-netwo...
There's also quantization. It's currently where a lot of the current grassroots research is happening, if you've seen the llama.cpp repo posted here on HN, it uses the original 16-bit float weights downsized to 4-bit integers with a comparable reduction in resource usage with relatively little performance loss.
The big players are also using quantization in hardware, most notably with Google's TPUs.
Having spent quite a bit of time playing around with llama.cpp, alpaca.cpp, loras, and the many other llama-based weights lately, here is my impression: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far. [0] If yo…
How does training on just 800k pieces of data need 7b parameters?
Having spent quite a bit of time playing around with llama.cpp, alpaca.cpp, loras, and the many other llama-based weights lately, here is my impression: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far. [0] If yo…
How does training on just 800k pieces of data need 7b parameters?
Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.
I wonder if when you train a model based on the weights of another model, do the owners of the original model have any sort of copyright claim? They trained it on data on which they had no copyright claim either, fwiw.
Are google and oracle are still fighting over java?
There's probably not a lot of hope for the litigation on hyper niche bits of config.
Is there a regularly updated repository containing all the releases of LLMs as they happen? TBH I am tired of having to doommark (doom-bookmark) so many repositories and links...Would appreciate some collected database.
Having spent quite a bit of time playing around with llama.cpp, alpaca.cpp, loras, and the many other llama-based weights lately, here is my impression: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far. [0] If yo…
Can this potentially do that?
Ideally I’d like to have it generate descriptions of large amounts of code but would rather not burn tokens and lose privacy via OpenAI api. But I’d gladly keep a high end GPU burning on such task , even if that was actually slightly more expensive.
Edit: To clarify I do this partially now on batches of code via openAI api currently. It’s around 1-3 cents for a typical 400-800 line source code file. And I don’t mean feeding the full code base in as a single input.