Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

github.com

Post reply on HN
Print view View original

131–140 of 146 posts

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#131

Post by JackFr »

afro88 wrote: post #98
adam_arthur wrote: post #78

Earlier quoted context omitted.

Pretty cool! Honestly even if it were to take a few minutes per response, that's likely sufficient for many use cases. I'd get value out of that if it allowed bypassing a paywall. I'm curious how these models end up being monetized/supported financially, as they sound expensive to run at scale. The required disk space seems the biggest barrier for local.

If it's a few minutes per token you might be waiting a lot longer for a full response: https://blog.quickchat.ai/post/tokens-entropy-question/ I also wonder how open.ai etc provides access to these for free. Reminds me of the adage from when Facebook rose to popularity: "if something is free, 'you' are the product". Perhaps to gather lots more conversational training data for fine tuning.

It would be remarkable and surprising if they weren’t doing that.

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#132

Post by sillysaurusx »

laughingman2 wrote: post #128
sillysaurusx wrote: post #105

Earlier quoted context omitted.

Exactly! That was the exact thing I was trying to think of a way to do. Got any ideas? There’s discriminators, but after reading over prior work, it seems like they help, but they weren’t really a groundbreaking / effective solution. I had two harebrained ideas in mind. One is to add yolo style object detection. The difference between a blurry mess and a recognizable object is the fact that it’s a recognizable object…

Have you checked RSSM approach in DreamerV1,V2,V3,PlaNet? It uses deterministic (GRU hidden state) and discrete stochastic latent states. The deterministic and stochastic (sampled) latent state are used to predict the next state. I think the stochastic state might help with your problem a bit.

Dear mystery HN’er, thank you so much. I hadn’t heard about RSSM, and your explanation was wonderfully helpful.

Much appreciated. Have a great weekend :)

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#133

Post by justplay »

JellyBeanThief wrote: post #90
adam_arthur wrote: post #78

Earlier quoted context omitted.

Pretty cool! Honestly even if it were to take a few minutes per response, that's likely sufficient for many use cases. I'd get value out of that if it allowed bypassing a paywall. I'm curious how these models end up being monetized/supported financially, as they sound expensive to run at scale. The required disk space seems the biggest barrier for local.

Crowd-funded AI training coming soon to Patreon?

do it now

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#134

Post by ck2 »

visarga wrote: post #117
ck2 wrote: post #79

Earlier quoted context omitted.

Datasets. The one with the largest, most personal, most obtrusive, invasive dataset will probably win. The one that has absorbed every podcast, every youtube video, every close-caption text in existence, will have the most "complete" answers.

Hidden datasets can be replaced with model predictions collected from a public API. So they can be "exfiltrated" from the trained model. And we already maxed out on the accessible online text and the good quality sources. What is going to make a difference is running models to generate more text for training, because relying on humans alone doesn't scale. For example we could be using LLMs to do brute force problem s…

How about illegal datasets like all the phone calls the NSA has been collecting domestically? Someone is going to train a private ChatGPT with that for queries.

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#135

Post by euclaise »

solomatov wrote: post #50
aljungberg wrote: post #20

THe RWKV model seems really cool. If you could get transformer-like performance with an RNN, the “hard coded” context length problem might go away. (That said, RNNs famously have infinite context in theory and very short context in reality.) Is there a primer for what RWKV does differently? According to the Github page it seems the key is multiple channels of state with different decaying rates, giving I assume, a co…

My understanding is that RNNs aren't worse than Transformers per se, they are just slower to train, and use GPU much more efficiently, i.e. much more stuff could be run in parallel.

We also don't have evidence that they scale the way transformers do

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#137

Post by anigbrowl »

dragonwriter wrote: post #94
anigbrowl wrote: post #73

Earlier quoted context omitted.

How much of this is the language vs the vast amount of passably accurate domain knowledge? ChatGPT etc. seem magic because they can answer questions about virtually anything with a high degree of plausibility. It often gets specific facts wrong, but the general contours are correct. Many of us know a lot of trivia/specialist knowledge, but I don't think anyone is as broadly informed as ChatGPT appears to be. It's not…

> How much of this is the language vs the vast amount of passably accurate domain knowledge? LLMs don’t have domain knowledge, its all language.

That's what I meant by 'It's not clear where the language ends and the encyclopedic knowledge starts,' since the model (and perhaps our brains) make little distinction.

But the model seems to be storing an absolutely vast amount of information, beyond the the capability of any individual person to accumulate and recall. This is clearly not a prerequisite for language, even if the information is represented linguistically. Put another way, at age 20 I had read maybe 10-20% of what I've read since, but I was capable of reading comprehension and conversation even though my levels of knowledge and insight were much lower. By 'comprehension' I mean in the sense of being able to read a piece of text and answer questions about it or rewrite it, without necessarily having any priors about the topic; the kind of task we expect to be able to assign to a high school graduate.

I'm wondering what the size of an 'ignorant' language model is, as a precursor to more curated/directed training. While the state of the art is very impressive, it's a bit like taking a feast for a thousand people and rendering it into a giant cube of spam. This strategy seems guaranteed to produce a succession of increasingly capable idiots savant but limits other avenues of exploration.

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#139

Post by adam_arthur »

5e92cb50239222b wrote: post #101
adam_arthur wrote: post #71

Earlier quoted context omitted.

In 20 years I'll still be alive and enjoying myself. Does 20 years seem long to you?

That's one hell of an assumption. Many of my Russian friends were absolutely certain they'll be alive and well for at least the next 20 years not that long ago.

If I'm dead then being able to run an AI locally doesn't matter anyway

Re: ChatRWKV, like ChatGPT but powered by the RWKV (RNN-based, open) language model

#140

Post by corobo »

ggerganov wrote: post #93
corobo wrote: post #70

Earlier quoted context omitted.

How badly is bad? What sort of output are we talking? I am asking as I once had a Markov-chain IRC bot* and while it often struggled to string together a sentence, it was quite hilarious sometimes. Absolutely pointless other than the occasional laugh. Can it form sentences or are those small models completely unusable for anything? I'm not thinking OpenAI level uses - sort of compare a Postgres cluster to a SQLite fi…

I was recently playing with the GPT-2 and GPT-J models. Results are often non-sensical for any practical purposes, but I think can be used for making something fun - similar to your IRC bot idea. If you are interested in running these models yourself without having a beefy GPU, you can try my custom inference implementation. It's in pure C/C++ without any 3rd party dependencies, runs straight on the CPU and builds ve…

I'll have to have a look into those, there's an audience of about 12 people that would be thrilled to hear "eggbot" is back with a shiny modern brain on, haha.

Oddly enough any processing delay is good in an "AI" chat bot, within reason, makes it feel more natural rather than getting a response ping instantly. Chat version of uncanny valley or something, haha.

Something it also did in Markov form was pick randomly from the longest words in the sentence it had decided to reply to, build the rest of it from that, then run itself "backwards" from the picked word to a sentence starter word it knew.

Thank you for the reply! Looking forward to some tinkering.

Post reply on HN