Earlier quoted context omitted.
Can confirm, at least on Gpt4all. Just tried it and it's nothing like ChatGPT
ChatGPT is GPT3.5(++)
If you pay you can use GPT-4 within the ChatGPT UI.
Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
251–260 of 314 posts
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#252Having spent quite a bit of time playing around with llama.cpp, alpaca.cpp, loras, and the many other llama-based weights lately, here is my impression: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far. [0] If yo…
This sentence defies lay people: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far.
Less garbage in, less garbage out?
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#253Earlier quoted context omitted.
According to the US Copyright office's recent ruling, nothing generated by these models can be copyrighted (!), so in theory you should be fine!
Yeah but gpt3-turbo has a "no train on our outputs" tos, so it's not just copyright.
Does it actually? The only thing I ever saw was "don't use our models to build competing models". Which is clearly ambiguous at best when dealing with generative natural text. This also implies to me that using their models to help with fine tunes of their other offerings would be ok.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#254Another dumb question: is it possible to "fine-tune" this model with a "corpus" of specialized data/jargon in another language ? I tried to do this with OpenAI API in order to improve the response on a specialized field (pediatric oncology) in another language (Italian) but the results were mixed (not to say disappointing).
I imagine the path to doing this, assuming the model already has good Italian, is to train it on all educational materials that go into getting a degree in pediatric oncology in Italian. This would help the model to actually understand the jargon, by tying it together with its baseline knowledge of Italian.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#255Earlier quoted context omitted.
How are the llama weights usable by the public? Even if leaked, doesn't using it count as piracy and thus a violation of either copyright or database laws?
It's not at all clear whether weights are copyrightable.
I'm pretty sure they are. If not copyrightable, then at least the database law should apply. One can easily make the case in front of a judge that the situation is similar to databases: the value of weights lies in the amount of work needed to gather the training data, thus weights should be considered a sort of crystallization of a database.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#256Earlier quoted context omitted.
This sentence defies lay people: The biggest deal with this isn't the published lora adapter (which seems limited to llama 7b), but the cleaned training data, which is likely better than the previous data sets used to train the alpaca-inspired loras that have been publicly released so far.
I have casually followed countless different news cycles on various complicated tech topics over my decades long career. I can't recall a single one that has consistently made me feel like an idiot more than how people talk about this recent AI wave. There just seems to be so much more jargon involved in this subject that makes casual perusing of the latest developments impenetrable.
The AI Today glossary and podcast series has been helpful for a grounding on basic concepts: https://www.aidatatoday.com/aitoday/
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#257The weights downloaded super slow for me, so I made a torrent of them:
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#258Earlier quoted context omitted.
I agree. I think that's why it's important that the lesson isn't "if you give an inch, the community will take a mile." Otherwise, they will keep it behind closed doors.
Even if they wanted to keep it behind close doors, it could easily get leaked.
[deleted]
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#259Anyone see how the binaries are generated? Seems like they are committed directly to the repo without any way of reproducing them locally? Seems strange.
Re: Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa
#260Earlier quoted context omitted.
There's a neat trick when you encounter jargon. 1. Identify the jargon terms you don't understand 2. Lookup papers that introduce the jargon terms 3. Skim-read the paper to get the gist of the jargon If you don't want to do this, then you don't have to feel uneducated. You can simply choose to feel like your time is more important than skimming a dozen AI papers a week. But for example, here's what I did to understan…
> 3. Skim-read the paper to get the gist of the jargon Or, you know, you could ask ChatGPT to explain it to you... Granted the term was coined 2021>=. Even if it wasn't but the paper is less than 32k tokens... 0.6c for the answer doesn't seem all that steep. edit: grammar
No need to pay yourself. Uploaded https://arxiv.org/pdf/2106.09685.pdf to scisummary:
This text discusses various studies and advancements in the field of natural language processing (NLP) and machine learning. One study focuses on parameter-efficient transfer learning, and another examines the efficiency of adapter layers in NLP models. Further studies evaluate specific datasets for evaluating NLP models. The article proposes a method called LoRA (low rank adaptation) for adapting pre-trained neural network models to new tasks with fewer trainable parameters. LoRA allows for partial fine-tuning of pre-trained parameters and reduces VRAM usage. The article provides experimental evidence to support the claims that changing the rank of Delta W can affect the performance of models, and that LoRA outperforms other adaptation methods across different datasets. The authors propose LoRA as a more parameter-efficient approach to adapt pre-trained language models to multiple downstream applications.