81–90 of 314 posts
I feel bad - now someone has to pay 9 cents * 4GB in S3 costs when I downloaded the weights. Cloudflare R2 has free bandwidth and is much faster (my current speed for any S3 download is just 10MBit despite 300MBit connection) - way to few devs use R2.
Earlier quoted context omitted.
I’ll ask a dumb question. On another of the numerous LLM related posts I was asking if any of the self host-able open model can do code summaries at close to the quality of GPT 3.5 turbo. I was basically told nowhere close yet. Can this potentially do that? Ideally I’d like to have it generate descriptions of large amounts of code but would rather not burn tokens and lose privacy via OpenAI api. But I’d gladly keep a…
We're a long long long way off that. So check back in two months. Jokes aside, the limiting factor will be either a technique to pack all of the code into smaller tokens like semantic search (someone else will be able to comment on this as that's the limit of my understanding) or GPU memory for input tokens. Buying a "high end GPU" isn't buying a 4090 or even two, it's 250k on a DGX unit and putting it in a datacentr…
did anyone get this to work without running their weird binary programs? I mean, it looks like it's supposed to be Python, but at least the `generate.yaml` file doesn't have the right content to work like they imply it should..
Earlier quoted context omitted.
I’ll ask a dumb question. On another of the numerous LLM related posts I was asking if any of the self host-able open model can do code summaries at close to the quality of GPT 3.5 turbo. I was basically told nowhere close yet. Can this potentially do that? Ideally I’d like to have it generate descriptions of large amounts of code but would rather not burn tokens and lose privacy via OpenAI api. But I’d gladly keep a…
We're a long long long way off that. So check back in two months. Jokes aside, the limiting factor will be either a technique to pack all of the code into smaller tokens like semantic search (someone else will be able to comment on this as that's the limit of my understanding) or GPU memory for input tokens. Buying a "high end GPU" isn't buying a 4090 or even two, it's 250k on a DGX unit and putting it in a datacentr…
I haven't started to play with LLMs locally in anger yet, but I was under the impression that you could use a 4090 in combination with FlexGen to achieve this rather than having to buy special hardware?
Earlier quoted context omitted.
I wonder if when you train a model based on the weights of another model, do the owners of the original model have any sort of copyright claim? They trained it on data on which they had no copyright claim either, fwiw.
As I understand it, it's not a copyright issue, it's a terms-of-service issue. You're only authorized to use GPT in accordance with their terms, which prohibit using the service to develop a competitor. Doing so anyway is unauthorized use of the service, which is a Computer Fraud and Abuse Act violation, independent of what is and isn't copyrightable.
The important takeaway for both GPT4All and Alpaca is that once an expensive proprietary model is released, people can easily train cheaper OSS models on input/output pairs. ⇒ LLMs are not defensible ⇒ LLMs will become commoditized ⇒ Prices will drop ⇒ Great for open source
Wow things are moving super fast, but it's scary that everything is being tainted by licensing. Super fun for research, but commercial endeavors are on shady grounds.
Hopefully someone will be brave enough to fight Meta on the LLaMA copyright issue. My concern is that the people who can afford a massive legal battle can also afford to train their own model.
Earlier quoted context omitted.
A Lora is a layer on top of a model, the big deal isn’t that this exists (it’s a Lora for the weakest llama), but the fact they shared their dataset. The stronger llamas trained with this data will produce even better Lora’s and better results.
What is a lora or llama? Google gives me nothing.
lora: efficient way of fine-tuning a model like llama, where instead of recreating an entire model, you're keeping the base model and generating a fine-tunings file to apply on top of it.
toaster: any machine with like 4GB of RAM available to fit the model
This looks like a fork of alpaca.cpp, which is a fork of llama.cpp. I noticed it doesn't seem to support the ggml weights released with those two.