Earlier quoted context omitted.
It’s shared context for anyone familiar with the field.
Amusimgly, also how cults and fascism works.
They also breathe air. What’s your point? Don’t exercise reading comprehension because bad people do it too?
LLMs are real, AI is fake
41–43 of 43 posts
Re: LLMs are real, AI is fake
#42Earlier quoted context omitted.
> Err, no? That's not at all how llms work. The Transformer architecture that almost all LLMs use are composed of many layers in sequence, each containing an attention component and a neural network component. The attention component copies data between tokens/vectors in the current context, and the neural network adjusts each individual vector in the context. Notably, the neural network component behaves like a comp…
>Notably, the neural network component behaves like a compressed index of the training data. The "Compression" part is indeed about the model finding order in the training data. But this is not about applying some predefined compression algorithm on it, but by actually learning an algorithmic representation of the data. This has nothing to do with "consulting the training data", because the training data cannot be re…
great distinction. can we use it to freely produce and share mp3/h264 copies of any media? because it is provably impossible to reconstruct original based on that information.
Re: LLMs are real, AI is fake
#43Earlier quoted context omitted.
>Notably, the neural network component behaves like a compressed index of the training data. The "Compression" part is indeed about the model finding order in the training data. But this is not about applying some predefined compression algorithm on it, but by actually learning an algorithmic representation of the data. This has nothing to do with "consulting the training data", because the training data cannot be re…
great distinction. can we use it to freely produce and share mp3/h264 copies of any media? because it is provably impossible to reconstruct original based on that information.
Well, that is what video models do.
But keep in mind: So far there is no way to train the models while completely avoiding memorization and only including generalization. That would be a great way to avoid any copyright issues, but all attempts I have seen so far were fairly limited.