Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

Post by gary_0 » Tue, Mar 28, 2023, 9:30 PM UTC

fnordpiglet wrote: post #142

gary_0 wrote: post #126

Earlier quoted context omitted.

That's how the brain evolved, after all: take incoming input, process according to previous patterns, output a reaction to peripheral nervous system that will enhance survival. The evolutionary purpose of a cortex is to store a statistical model of an animal's environment based on past stimulus. Human intelligence is just a complicated permutation of these initial systemic parameters.

That’s why people have to take courses in reasoning and logic to be able to successfully apply it. It’s not like reasoning and logic are natural human skills. They’re learned through reinforcement learning by repeatedly exposing our brain to patterns again and again until we can train our stochastic expectation juice to not hallucinate an answer.

Post by layer8 » Tue, Mar 28, 2023, 9:38 PM UTC

gary_0 wrote: post #151

fnordpiglet wrote: post #142

Earlier quoted context omitted.

That’s why people have to take courses in reasoning and logic to be able to successfully apply it. It’s not like reasoning and logic are natural human skills. They’re learned through reinforcement learning by repeatedly exposing our brain to patterns again and again until we can train our stochastic expectation juice to not hallucinate an answer.

And it takes ~20 years to train a new brain so it can coherently answer questions about a wide variety of topics. Even worse, you can't even copy-paste it once you're done!

Post by layer8 » Tue, Mar 28, 2023, 9:43 PM UTC

mcbuilder wrote: post #139

tjr wrote: post #131

Earlier quoted context omitted.

Not exactly the same, but from: https://www.instagram.com/p/CQdBiVyh5C2/?hl=en Now that the cat is out of the bag, or, should I say the confetti is out of the… can?

Is it strange that I think it's more likely that LLMs generalize and synthesize new things out of old patterns instead of just copying a singular example.

Post by layer8 » Tue, Mar 28, 2023, 9:50 PM UTC

jazzkingrt wrote: post #34

2bitencryption wrote: post #26

This type of article (or press release, or whatever you want to call it) is exactly what makes the future so interesting. The cat is out of the bag, the genie is out of the bottle, the confetti has left the cannon[0]. It's tempting to see a world dominated by Google Bard, ChatGPT, Bing Search, etc. And no doubt, they will be huge players, with services that are far more powerful than anything that can be run on the e…

Serious question: is it typical to describe client-side computing as "on the edge"? I thought running something on the edge referred to running it in close network proximity to the user, rather than users having control and running things themselves.

Post by IshKebab » Tue, Mar 28, 2023, 9:58 PM UTC

brucethemoose2 wrote: post #18

FYI: Cerebras's nodes are very different than your typical Nvidia training nodes: https://www.anandtech.com/show/16626/cerebras-unveils-wafer-... Each individual "chip" has 40GB of SRAM vs ~76MB for the Nvidia H100, and networked pools of external RAM, SSDs and such. Thats why the training architecture is so different.