Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

cerebras.net

Post reply on HN
Print view View original

81–90 of 231 posts

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#81

Post by cs-fan-101 »

antimatter15 wrote: post #77

Looking at their charts it seems like their 6.7B model is considerably worse than GPT-J which is an existing open 6B model from several years ago. I wish rather than stopping training early they would have run more data through a small model so we could have something more competitive with LLaMA 7B.

Someone posted this repost from the Cerebras Discord earlier, but sharing for visibility -

"We chose to train these models to 20 tokens per param to fit a scaling law to the Pile data set. These models are optimal for a fixed compute budget, not necessarily "best for use". If you had a fixed parameter budget (e.g., because you wanted to fit models on certain hardware) you would train on more tokens. We do that for our customers that seek that performance and want to get LLaMA-like quality with a commercial license"

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#82

Post by wsgeorge »

TeMPOraL wrote: post #75
wsgeorge wrote: post #42

Earlier quoted context omitted.

I believe this has been extended to mean "on device", which is interesting. See Gerganov's article on Github [0]. I wrote about this here [1] where I made a contrast between the core and the edge. I think the term maps well to this meaning. What I find more interesting is that in the classic "close network proximity", some parts of the world may not have benefited as much from that trend since the closest nodes of a…

> I believe this has been extended to mean "on device", which is interesting. I don't like the connotations this carries. This is almost openly talking about reaching all the way into peoples' hardware to run your software, for your benefit, on them, without their knowledge, consent or control...

I see. Hadn't considered this. Yes, I see how that might be a concern.

What I think is important in this AI Spring is that we make it possible for people to run their own models on their own hardware too, without having to submit anything to a large, centralised model for inference.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#83

Post by meghan_rain »

vintermann wrote: post #52
bogwog wrote: post #47

Earlier quoted context omitted.

In other words, they’re actually incentivized to help make LLMs as accessible as possible, rather than try to keep them locked up to hide them from competitors. Which makes me wonder if Nvidia is doing anything with LLMs too?

NVidia has certainly pushing the envelope on image generation. StyleGAN3 was really cool when it came. But it is an issue that their chips are hardly optimized for LLMs.

How can a GPU be optimized for StyleGAN but not LLMs? Serious question.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#84

Post by examplary_cable »

wg0 wrote: post #45

Noob to ML in practice. These models containing weights, all of them, do they have a standard file/binary format?

[I'm not an expert] but I believe .ckpt and .safetensors.

The problem with .ckpt is that it executes arbitrary code in your machine(very unsafe). While .safetensors was made by huggingface in order to have a safe format to store the weights. I've also seen people load up the llama 7B via a .bin file.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#85

Post by hiAndrewQuinn »

2bitencryption wrote: post #26

This type of article (or press release, or whatever you want to call it) is exactly what makes the future so interesting. The cat is out of the bag, the genie is out of the bottle, the confetti has left the cannon[0]. It's tempting to see a world dominated by Google Bard, ChatGPT, Bing Search, etc. And no doubt, they will be huge players, with services that are far more powerful than anything that can be run on the e…

I for one dream of a future without maps. I want to walk through a distant forest to find an ancient, unconnected ESP-32 in the bark of a tree containing a tiny specialized AI that can only tell me about things relevant to the area, how far to walk upstream to the nearest town. And only if I can find it and scan an RFID tag to wake it up.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#86

Post by barking_biscuit »

simon83 wrote: post #72
2bitencryption wrote: post #26

This type of article (or press release, or whatever you want to call it) is exactly what makes the future so interesting. The cat is out of the bag, the genie is out of the bottle, the confetti has left the cannon[0]. It's tempting to see a world dominated by Google Bard, ChatGPT, Bing Search, etc. And no doubt, they will be huge players, with services that are far more powerful than anything that can be run on the e…

Google: "confetti has left the cannon" > No results found for "confetti has left the cannon". I'm amazed that a "stochastic parrot" can come up with such a beautiful idiom.

Out of distribution generations are a thing.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#87

Post by option »

bogwog wrote: post #47
mometsi wrote: post #14

Summary: This is a company that makes AI accelerator ICs. They reimplemented Chinchilla and released the model weights under a permissive license.

In other words, they’re actually incentivized to help make LLMs as accessible as possible, rather than try to keep them locked up to hide them from competitors. Which makes me wonder if Nvidia is doing anything with LLMs too?

obviously yes, there are larger (20B) models trained on more tokens that are released by NVIDIA under commercial-friendly license on HF

https://huggingface.co/nvidia/nemo-megatron-gpt-20B

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#88

Post by option »

amilios wrote: post #64

Comparing the 13B model here https://huggingface.co/cerebras/Cerebras-GPT-13B to LLaMA-13B https://github.com/facebookresearch/llama/blob/main/MODEL_CA... you can see that in all of the reasoning tasks Cerebras-GPT lags behind. Any reason to use Cerebras instead of LLaMA? Doesn't seem like it.

it lags behind because according to their blogpost it was trained on <300B tokens. LLaMAs as far as I know were trained on more than trillion

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#89

Post by barking_biscuit »

akiselev wrote: post #68
lioeters wrote: post #50

Earlier quoted context omitted.

Yes, yes, and yes. I'm waiting for an actually open AI that can run on the edge, purely on commodity hardware like our laptops and phones - it's inevitable. I imagine this "cat out of the bag" situation, the democratization and commodification of powerful technology accessible and affordable to the public, is similar to what's happening with single-board computers and microcontrollers like Raspberry Pi, Arduino, ESP3…

After using ChatGPT 4 extensively for a few days, I think we're probably only a few years away from the first generation of truly conversational assistants ala Jarvis in Iron Man. Between LangChain and existing voice recognition software, we've already 95% of the way there, it just needs to be packaged up into a UI/UX that makes sense. These local models are absolutely critical for that to happen though. I'm hitting…

Only a few years? Nobody can predict accurately in years anymore. Feels more like "only a few months" away.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#90

Post by visarga »

Of course this is great news, I hope these models can be fine-tuned to be like lighter versions of chatGPT. But I remember reading in the LLaMA paper that a small model can still improve when trained more than the Chinchilla budget.

> For instance, although Hoffmann et al. (2022) recommends training a 10B model on 200B tokens, we find that the performance of a 7B model continues to improve even after 1T tokens.

Cerebras says:

> For instance, training a small model with too much data results in diminishing returns and less accuracy gains per FLOP

But this is only of concern when you care about the training cost, such as when you are budget limited researcher or a company who doesn't deploy models at scale. But when you care about the total cost of deployment, then making a small model even better with lots of data is a smart move. In the end it matters more to have the most efficient model in prediction, not the most efficient model in training.

Post reply on HN