ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
1–10 of 146 posts
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#2[deleted]
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#3This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#4This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.
The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person.
Suddenly it became something 'real' then.
(This is purely talking about the public popularity of GPT)
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#5This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.
The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)
One of the important parts of ChatGPT over plain GPT-3 is the reinforcement learning from human feedback to ensure alignment, without which it's not quite as good of a product for the public.
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#6For those wondering how on earth they are getting decent results from a RNN without long range forgetting, I don't really know either!
But they reference https://arxiv.org/abs/2105.14103 and the bottom section of https://github.com/BlinkDL/RWKV-LM has an explainer.
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#7This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.
The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)
This is a remarkably good take that just didn't dawn on me until I read your comment. Even if ChatGPT had a lesser quality than the current iteration, the fact that they had a way for anyone to easily interact with it really was a homerun, snd can be for any software, really.
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#8This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.
The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)
They also did reinforcement learning on top of a frozen trained model. It is considerably more than just attaching a UI as that would just finish sentences compared to answering questions. https://huggingface.co/blog/rlhf
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#9This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.
The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)
No. ChatGPT’s UI is incredibly simple and basically exactly what ever chat bot test repl looks like.
The delta of GPT3 -> ChatGPT is from the expanded context and control the model offers through fine tuning. Eg read the instructgpt paper to see the path on the way to ChatGPT.
Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model
#10This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.
The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)
It's not just the UI. ChatGPT (which is further finetuned and uses RLHF) definitely produces better output than GPT3, especially without prompt engineering.