Earlier quoted context omitted.
NeoX 20B is a fantastic open source model.
It's nice, but a far cry from gpt-3
NLP Cloud has a finetuned version of neoX which works incredibly well.
Open source solution replicates ChatGPT training process
11–20 of 158 posts
Re: Open source solution replicates ChatGPT training process
#12Finally, an open-source equivalent to ChatGPT emerging out of the AI euphoria will begin to extinguish the hype out of OpenAI's ChatGPT moat, just like how GPT-3 and DALLE-2 were almost immediately disrupted by open-source models as well. This (and other open-source AI models), not 'ChatGPT', 'DALLE-2', etc is what will change the AI landscape for everyone, permanently forever.
Is there a GPT-3 disruptor? All the open sourced models are GPT2 improvements, and GPT2 was open sourced by OpenAI. GPT3/4 is simply too expensive for consumer GPUs, any open sourced versions will have to run on A100s in the cloud, so by nature centralized. Granted, having multiple providers also counts as removing the moat. But BLOOM for example (An attempt at replicating GPT3), no one actually uses. Because its sim…
Dalle 2 has also been disrupted because OpenAI has heavily nerfed the model, probably by greatly reducing the steps in the upscaler models (Dalle 2 uses diffusion-based upscaler models and therefore very expensive to run), so the images have good coherence but really bad texture, full of artifacts, ironically since the GAN models had the opposite result, very bad coherence and good texture; also OpenAI has introduced very few features and there is no way to finetuned the model as with GPT-3.
Meanwhile, the MJ model outputs extremely good images and SD can be conditioned, fine-tuned, etc. in a really versatile way and extremely good quality (if you know what you are doing).
Re: Open source solution replicates ChatGPT training process
#13Finally, an open-source equivalent to ChatGPT emerging out of the AI euphoria will begin to extinguish the hype out of OpenAI's ChatGPT moat, just like how GPT-3 and DALLE-2 were almost immediately disrupted by open-source models as well. This (and other open-source AI models), not 'ChatGPT', 'DALLE-2', etc is what will change the AI landscape for everyone, permanently forever.
[deleted]
Re: Open source solution replicates ChatGPT training process
#14Finally, an open-source equivalent to ChatGPT emerging out of the AI euphoria will begin to extinguish the hype out of OpenAI's ChatGPT moat, just like how GPT-3 and DALLE-2 were almost immediately disrupted by open-source models as well. This (and other open-source AI models), not 'ChatGPT', 'DALLE-2', etc is what will change the AI landscape for everyone, permanently forever.
I, for one, would like to see an open-source model similar to Stable Diffusion, but for text. It would be a great way to empower general folk without having to pay OpenAI, and not have to worry about the LLM's belief system, which is conservative-biased in the case of ChatGPT[1] (HN discussion[2]). [1] https://davidrozado.substack.com/p/openaicms [2] https://news.ycombinator.com/item?id=34625001
[deleted]
Re: Open source solution replicates ChatGPT training process
#15Finally, an open-source equivalent to ChatGPT emerging out of the AI euphoria will begin to extinguish the hype out of OpenAI's ChatGPT moat, just like how GPT-3 and DALLE-2 were almost immediately disrupted by open-source models as well. This (and other open-source AI models), not 'ChatGPT', 'DALLE-2', etc is what will change the AI landscape for everyone, permanently forever.
Yeah, for the worse. We will have a ton of bullshit at scale. And the web will be done for.
I hope the arms race makes us smarter. We're going to need AI to sift through all the BS. My hope is that once we're drowning in deepfakes daily, the average user will come to the conclusion that they can't believe stuff they see, and will realize neither what the read nor hear. The transition will be rough.
Re: Open source solution replicates ChatGPT training process
#16How good is the quality of this? BLOOM is a 176B parameter model, but it doesn't seem to compare to GPT-3 (175B parameters) in terms of output quality.
It's because BLOOM is undertrained, you can prune a lot of weights in BLOOM and it doesn't impact performance. Look at Chinchilla paper[1], 70B model outperforms 175B GPT-3 model.
Re: Open source solution replicates ChatGPT training process
#17Is the term "ChatGPT" being used in place of GPT-3 here? Is this thing actually replicating the GPT-3 training process?
The thing that makes ChatGPT interesting (over regular GPT-3) is the RLHF process, but this article doesn't seem to touch on that at all, unless I've missed something.
Re: Open source solution replicates ChatGPT training process
#18Finally, an open-source equivalent to ChatGPT emerging out of the AI euphoria will begin to extinguish the hype out of OpenAI's ChatGPT moat, just like how GPT-3 and DALLE-2 were almost immediately disrupted by open-source models as well. This (and other open-source AI models), not 'ChatGPT', 'DALLE-2', etc is what will change the AI landscape for everyone, permanently forever.
[deleted]
Re: Open source solution replicates ChatGPT training process
#19"hitting 100 million monthly active users 2 months after its launch".
I'm deeply suspicious of that number. It came from Similarweb, who track these things through analytics gathered from browser extensions.
I trust this article more: https://www.nytimes.com/2023/02/03/technology/chatgpt-openai...
"But two months after its debut, ChatGPT has more than 30 million users and gets roughly five million visits a day, two people with knowledge of the figures said."
"Two people with knowledge of the figures" is journalism speak for "I heard this off the record from people with insider info, and I'm ready to report it because those two different sources provided the same number".
Re: Open source solution replicates ChatGPT training process
#20Finally, an open-source equivalent to ChatGPT emerging out of the AI euphoria will begin to extinguish the hype out of OpenAI's ChatGPT moat, just like how GPT-3 and DALLE-2 were almost immediately disrupted by open-source models as well. This (and other open-source AI models), not 'ChatGPT', 'DALLE-2', etc is what will change the AI landscape for everyone, permanently forever.
"just like how GPT-3 ... immediately disrupted by open-source models as well."
Which open source alternatives to GPT-3 have you seen that most impressed you?
I've not yet found any that are remotely as useful as GPT-3, at least for the kinds of things I want to use them for (generating SQL queries from human text, summarizing text, that kind of thing)