Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

github.com

Post reply on HN
Print view View original

11–20 of 146 posts

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#11

Post by gamegoblin »

phist_mcgee wrote: post #4
totoglazer wrote: post #3

This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.

The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)

This is mostly correct. GPT3.5 is better, has a larger context window, etc. But it's a very incremental step above GPT3.

I had wired up GPT3 to a Twilio phone number and made something basically like ChatGPT months before ChatGPT was released -- me and my friends texted it all the time to get information, similar to how people use ChatGPT. The prompt to get decent performance is super simple. Just something like:

    The following is a transcript between a human and a helpful AI assistant.
    The AI assistant is knowledgeable about most facts of the world and provides concise answers to questions.

    Transcript:
    {splice in the last 30 messages of the conversation}

    The next thing the assistant says is:
Over time I did upgrade the prompt a bit to improve performance for specific kinds of queries, but nothing crazy.

Cost me $10-20/mo to run for the low/moderate use by me and a few friends.

Interestingly, for people who didn't know its limitations / how to break it, it was basically passing the turing test. ChatGPT is inhumanly wordy, whereas GPT3 can actually be much more concise when prompted to do so. If, instead of prompting it that it is an AI assistant, you prompt it that it is a close friend with XYZ personality traits, it does a very good job of carrying on a light SMS conversation.

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#12

Post by moffkalast »

phist_mcgee wrote: post #4
totoglazer wrote: post #3

This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.

The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)

Well yes, having no context memory, being slightly worse and requiring either a monster rig to run or paying per prompt made it completely and utterly irrelevant.

Even now that it's improved and free to use its actual practical usability is marginal at best given the rate of blatantly wrong info being spewed with 105% confidence at the moment.

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#13

Post by t3estabc »

phist_mcgee wrote: post #4
totoglazer wrote: post #3

This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.

The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)

[dead]

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#14

Post by merely-unlikely »

gamegoblin wrote: post #11
phist_mcgee wrote: post #4

Earlier quoted context omitted.

The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)

This is mostly correct. GPT3.5 is better, has a larger context window, etc. But it's a very incremental step above GPT3. I had wired up GPT3 to a Twilio phone number and made something basically like ChatGPT months before ChatGPT was released -- me and my friends texted it all the time to get information, similar to how people use ChatGPT. The prompt to get decent performance is super simple. Just something like: The…

I half remember one of Google’s many chat apps having an AI assistant a number of years ago (Allo maybe?)

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#15

Post by andai »

gamegoblin wrote: post #11
phist_mcgee wrote: post #4

Earlier quoted context omitted.

The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)

This is mostly correct. GPT3.5 is better, has a larger context window, etc. But it's a very incremental step above GPT3. I had wired up GPT3 to a Twilio phone number and made something basically like ChatGPT months before ChatGPT was released -- me and my friends texted it all the time to get information, similar to how people use ChatGPT. The prompt to get decent performance is super simple. Just something like: The…

>If [...] you prompt it that it is a close friend with XYZ personality traits

A couple years ago a friend and I trained GPT-2 on our WhatsApp chat history. GPT-2 was more primitive, but it still managed to capture the gist of our personalities and interests, which was equal parts amusing and embarrassing.

We'd have it generate random chats, or ask it questions to see what simulated versions of ourselves would say.

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#16

Post by b33j0r »

tinsmith wrote: post #7
phist_mcgee wrote: post #4

Earlier quoted context omitted.

The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)

This is a remarkably good take that just didn't dawn on me until I read your comment. Even if ChatGPT had a lesser quality than the current iteration, the fact that they had a way for anyone to easily interact with it really was a homerun, snd can be for any software, really.

My family told me that ChatGPT came up from the pulpit AT CHURCH

Me? I made a few comments like a scared luddite when ChatGPT solved two of my outstanding engineering problems instantly.

I got better. But this is exactly right. The world in general now knows about AI and ML. It’s a pivot point.

When something scares a seasoned engineer for a minute, and anyone can now make use of this… write it down in your diary as a moment in history

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#18

Post by leodriesch »

The readme does not seem to be geared towards people not familiar with the topic.

My questions:

- Is this on the run on consumer GPU scale, or run on 8 A100 scale or you can’t run it yourself ever scale? - How does it compare to other language models in quality/abilities? - What is the training data?

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#19

Post by axiom92 »

phist_mcgee wrote: post #4
totoglazer wrote: post #3

This might be an interesting language model. However people care about ChatGPT entirely due to its quality, which this doesn’t demonstrate yet.

The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person. Suddenly it became something 'real' then. (This is purely talking about the public popularity of GPT)

Some evidence to confirm this:

1. Twitter thread with examples: https://twitter.com/sjwhitmore/status/1601254826947784705

2. Tweet/screenshot + Colab notebook:https://twitter.com/aman_madaan/status/1599549721030246401, https://tinyurl.com/codex-chat-gpt

The second tweet is mine.

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#20

Post by aljungberg »

THe RWKV model seems really cool. If you could get transformer-like performance with an RNN, the “hard coded” context length problem might go away. (That said, RNNs famously have infinite context in theory and very short context in reality.)

Is there a primer for what RWKV does differently? According to the Github page it seems the key is multiple channels of state with different decaying rates, giving I assume, a combination of short and long term memory. But isn’t that what LSTMs were supposed to do too?

Post reply on HN