Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

cerebras.net

Post reply on HN
Print view View original

141–150 of 231 posts

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#141

Post by sebzim4500 »

gamegoblin wrote: post #5
eldenring wrote: post #3

> Trained using the Chinchilla formula, these models provide the highest accuracy for a given compute budget. I'm confused as to why 111 million parameter models are trained with the Chinchilla formula. Why not scale up the training data? If you're training smaller models, surely optimizing performance is better than optimizing total compute. Seems like a silly misunderstanding of the Chinchilla paper, but I'm sure I…

True. There was a good blog post published about this a few weeks ago: https://finbarr.ca/llms-not-trained-enough/ Money quote for those who don't want to read the whole thing: ''' When people talk about training a Chinchilla-optimal model, this is what they mean: training a model that matches their estimates for optimality. They estimated the optimal model size for a given compute budget, and the optimal number of t…

>Chinchilla implicitly assumes that the majority of the total cost of ownership (TCO) for a LLM is the training cost. In practice, this is only the case if you’re a researcher at a research lab who doesn’t support products (e.g. FAIR/Google Brain/DeepMind/MSR). For almost everyone else, the amount of resources spent on inference will dwarf the amount of resources spent during training.

I'm not so convinced, especially if people are doing multiple training runs for hyperparameter tuning, cleaning data, fixing bugs, etc.

I would be very interested in knowing what portion of OpenAI's compute budget is training. I would not be surprised if it was a significant minority.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#142

Post by fnordpiglet »

gary_0 wrote: post #126
fnordpiglet wrote: post #115

Earlier quoted context omitted.

It’s because they’re confused in thinking human intelligence isn’t learned stochastic expectation.

That's how the brain evolved, after all: take incoming input, process according to previous patterns, output a reaction to peripheral nervous system that will enhance survival. The evolutionary purpose of a cortex is to store a statistical model of an animal's environment based on past stimulus. Human intelligence is just a complicated permutation of these initial systemic parameters.

That’s why people have to take courses in reasoning and logic to be able to successfully apply it. It’s not like reasoning and logic are natural human skills. They’re learned through reinforcement learning by repeatedly exposing our brain to patterns again and again until we can train our stochastic expectation juice to not hallucinate an answer.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#143

Post by slowmovintarget »

2bitencryption wrote: post #26

This type of article (or press release, or whatever you want to call it) is exactly what makes the future so interesting. The cat is out of the bag, the genie is out of the bottle, the confetti has left the cannon[0]. It's tempting to see a world dominated by Google Bard, ChatGPT, Bing Search, etc. And no doubt, they will be huge players, with services that are far more powerful than anything that can be run on the e…

> Without services spying on your data. Without censorship on what the model can/cannot say. Because it's all local...

Wouldn't that be nice? It would also be contrary to all experience of the outcomes and pulls of corporations in modern society. The "local" LLMs will be on the fringe more than at the edge, because the ones that work the best and attract the most money will be the ones controlled by walled-garden "ecosystems."

I really hope it's different. I really hope there are local models. Actual personal assistants actually designed to assist their users and not the people that provide the access.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#144

Post by 2bitencryption »

yieldcrv wrote: post #129
2bitencryption wrote: post #26

This type of article (or press release, or whatever you want to call it) is exactly what makes the future so interesting. The cat is out of the bag, the genie is out of the bottle, the confetti has left the cannon[0]. It's tempting to see a world dominated by Google Bard, ChatGPT, Bing Search, etc. And no doubt, they will be huge players, with services that are far more powerful than anything that can be run on the e…

> And smart people will package them as "one click installers" for people who aren't tech-savvy. This is already happening. Any projects I can follow? Because I haven't seen any one click installers yet that didn't begin with "first install a package manager on the command line"

I was mostly referring to this project, which has some 1-click installers: https://github.com/oobabooga/text-generation-webui#alternati...

Though I have not tried those 1-click installers, instead I have been manually running it.

That project is based on the concept of this Stable Diffusion project: https://github.com/AUTOMATIC1111/stable-diffusion-webui

Which is a few months ahead (because the Stable Diffusion tech happened a few months earlier) and is definitely at a point where anyone can easily run it, locally or on a hosted environment.

I expect this "text-generation-webui" (or something like it) will be just as easy to use in the near, near future.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#146

Post by malshe »

simon83 wrote: post #72
2bitencryption wrote: post #26

This type of article (or press release, or whatever you want to call it) is exactly what makes the future so interesting. The cat is out of the bag, the genie is out of the bottle, the confetti has left the cannon[0]. It's tempting to see a world dominated by Google Bard, ChatGPT, Bing Search, etc. And no doubt, they will be huge players, with services that are far more powerful than anything that can be run on the e…

Google: "confetti has left the cannon" > No results found for "confetti has left the cannon". I'm amazed that a "stochastic parrot" can come up with such a beautiful idiom.

Now Google has one search result pointing to your comment :)

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#147

Post by yieldcrv »

slickdork wrote: post #132
yieldcrv wrote: post #129

Earlier quoted context omitted.

> And smart people will package them as "one click installers" for people who aren't tech-savvy. This is already happening. Any projects I can follow? Because I haven't seen any one click installers yet that didn't begin with "first install a package manager on the command line"

Not an llm but this 1 click installer for stable diffusion is literally a 1 click installer. Impressively works. https://github.com/cmdr2/stable-diffusion-ui

> In the terminal, run ./start.sh (or bash start.sh)

smh.

well, very close! its interesting that they just got Mac M1/M2 support at all, 2 weeks ago.

for SD I've been using DiffusionBee since maybe October last year.

I expect LLMs to have something in a few weeks, just want to know about it so I can tell other people that need it that way.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#148

Post by tlb »

2bitencryption wrote: post #105
simon83 wrote: post #72

Earlier quoted context omitted.

Google: "confetti has left the cannon" > No results found for "confetti has left the cannon". I'm amazed that a "stochastic parrot" can come up with such a beautiful idiom.

For the curious, here was the conversation I had: ME: What's another saying similar to: "The cat is out of the bag" or "The genie is out of the bottle"? chatgpt: Another similar saying is "the toothpaste is out of the tube." Like the other two expressions, this phrase conveys the idea that something has been revealed or unleashed, and it cannot be undone or put back into its original state. ME: Can you invent a new p…

Conspiracy theory: this idiom used to be widespread on the internet, but a shadowy cabal of AI maximalists erased millions of such expressions from search engines so GPT could appear to re-invent them.

It is a good expression though -- evocative but not gross or violent. You could imagine many less successful analogies to something ejecting something else.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#149

Post by worldsayshi »

johtso wrote: post #116
2bitencryption wrote: post #105

Earlier quoted context omitted.

For the curious, here was the conversation I had: ME: What's another saying similar to: "The cat is out of the bag" or "The genie is out of the bottle"? chatgpt: Another similar saying is "the toothpaste is out of the tube." Like the other two expressions, this phrase conveys the idea that something has been revealed or unleashed, and it cannot be undone or put back into its original state. ME: Can you invent a new p…

Not exactly sure why it would be surprising that it can come up with a convincing idiom when it can produce remarkably good _poetry_

I think the thrilling part is that it's a somewhat atomic concept that can somewhat convincingly be proven to not exist in the training data. While poetry is more impressive if it's as original it's harder to show that it's not just stitched together from the training data.

Re: Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models

#150

Post by aiappreciator »

gamegoblin wrote: post #5
eldenring wrote: post #3

> Trained using the Chinchilla formula, these models provide the highest accuracy for a given compute budget. I'm confused as to why 111 million parameter models are trained with the Chinchilla formula. Why not scale up the training data? If you're training smaller models, surely optimizing performance is better than optimizing total compute. Seems like a silly misunderstanding of the Chinchilla paper, but I'm sure I…

True. There was a good blog post published about this a few weeks ago: https://finbarr.ca/llms-not-trained-enough/ Money quote for those who don't want to read the whole thing: ''' When people talk about training a Chinchilla-optimal model, this is what they mean: training a model that matches their estimates for optimality. They estimated the optimal model size for a given compute budget, and the optimal number of t…

"the training cost is going to be dominated by the inference cost."

That's only true for general-mass-consumer models.

Companies may want to fine-tune/train their own models, which don't have that many users for their narrow use cases (possibly only internal staff), will find that training cost is a substantial chunk of the TCO

Post reply on HN