Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

cerebras.net

Post reply on HN
Print view View original

11–20 of 231 posts

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#11

Post by alchemist1e9 »

spi wrote: post #7
tombert wrote: post #2

Has anyone tried this? I have 96GB of GPU memory; will that be enough to run the biggest model?

I have not tried, but 96GB of GPU memory is plenty, for inference there should certainly be no issue. Their biggest model has 13B parameters, you should be able to run inference (float16) already with 32GB of memory. With 96GB of memory you should also be able to fine-tune it (possibly some tricks like gradient accumulation and/or checkpointing might be needed), but you have to be ready for many days of computation..…

> but you have to be ready for many days of computation...

I was thinking since we have API prices in tokens and now it looks like self hosted inference on high end GPUs for similar models. Then based on electricity prices there will be a self-hosted prices in tokens. Then how close are these already? What is the markup today from roughly the raw electricity cost that OpenAI has.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#12

Post by murkt »

whalesalad wrote: post #4

This “AI spring” is really snowballing with the crazy nouns and terminology. Alpaca, llama and now chinchilla??

Chinchilla actually came before alpaca and llama.

Every new variation of model gets some new name, just like every library gets a new name. There were all kinds of BERTs before - DistilBert, Roberta, SciBERT, Schmobert, Schmuber, etc. Many hundreds of them, I think.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#15

Post by jhbadger »

whalesalad wrote: post #4

This “AI spring” is really snowballing with the crazy nouns and terminology. Alpaca, llama and now chinchilla??

As mentioned, chinchilla is not part of this trend, and chinchillas are rodents. Alpacas and llamas are South American camelids (animals related to camels). So if additional names are needed, I would expect them to be vicuña and guanaco, as they are also in the group.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#16

Post by JamesCoyne »

Slightly off-topic:

I remember seeing news about the enormous chip Cerebras was/is selling (pdf https://f.hubspotusercontent30.net/hubfs/8968533/WSE-2%20Dat...).

Has there been any indication that the LLMs released in the last few months use exotic hardware like this, or is it all "standard" hardware?

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#18

Post by brucethemoose2 »

FYI: Cerebras's nodes are very different than your typical Nvidia training nodes:

https://www.anandtech.com/show/16626/cerebras-unveils-wafer-...

Each individual "chip" has 40GB of SRAM vs ~76MB for the Nvidia H100, and networked pools of external RAM, SSDs and such. Thats why the training architecture is so different.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#20

Post by simonw »

"Cerebras open sources seven GPT-3 models from 111 million to 13 billion parameters."

I don't understand why they describe them as GPT-3 models here as opposed to calling them GPT models. Or even LLMs - but I guess that acronym isn't as widely recognized.

Post reply on HN