11–20 of 173 posts
From the article: Pre-trained on a trillion "tokens"... Doesn't 7B indicates that it was trained on 7 billion tokens? Or am I misunderstanding the nomenclature?
Who writes this shit?
This is why it's not possible to slow down or "stop" AI: once the problems are solved the solutions turn out to be trivial to replicate. All it takes is compute.
This is why it's not possible to slow down or "stop" AI: once the problems are solved the solutions turn out to be trivial to replicate. All it takes is compute.
From the article: Pre-trained on a trillion "tokens"... Doesn't 7B indicates that it was trained on 7 billion tokens? Or am I misunderstanding the nomenclature?
No, 7B means it has 7 billion parameters.
From the article: Pre-trained on a trillion "tokens"... Doesn't 7B indicates that it was trained on 7 billion tokens? Or am I misunderstanding the nomenclature?
OpenAI is an eventually to be obsoleted initial brute force approach that will be abstracted over and over into a simpler code implementation with rules to recreate the old state.
kkrieger is a simple example of a tiny data model that can be deterministically rehydrated. It’s not unrealistic for AI models to become a seed value for a normalized code base to deterministically unpack into necessary electron state
Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…
This is why it's not possible to slow down or "stop" AI: once the problems are solved the solutions turn out to be trivial to replicate. All it takes is compute.
Governments have experience limiting the spread of digital content. For now at least, AI proliferation is not immune to those same tactics.
Hardly. I've played a lot with the 7,13, and 30B llamas as well as the 7 and 13B alpacas fine tuned by Stanford. They do not have emergent abilities like being able to generate rhymes or, say, represent a movie plot as emoji. Even openai's old text-davinci-003 (gpt3.5, but text completion, not the chat ones) far outperforms them. That said, I have hopes for a 65B 3-bit quantized alpaca-fine tuned. We'll see when some…
I am playing around with GPT-4 this week though. Let’s see how that goes.