71–80 of 231 posts
Comparing the 13B model here https://huggingface.co/cerebras/Cerebras-GPT-13B to LLaMA-13B https://github.com/facebookresearch/llama/blob/main/MODEL_CA... you can see that in all of the reasoning tasks Cerebras-GPT lags behind. Any reason to use Cerebras instead of LLaMA? Doesn't seem like it.
Can the LLaMA weights be used for commercial products?
This type of article (or press release, or whatever you want to call it) is exactly what makes the future so interesting. The cat is out of the bag, the genie is out of the bottle, the confetti has left the cannon[0]. It's tempting to see a world dominated by Google Bard, ChatGPT, Bing Search, etc. And no doubt, they will be huge players, with services that are far more powerful than anything that can be run on the e…
> No results found for "confetti has left the cannon".
I'm amazed that a "stochastic parrot" can come up with such a beautiful idiom.
Earlier quoted context omitted.
It’s unbelievable stuff. Does anyone know how much a single box costs? They are selling them it looks like.
CS-1 costs "$2-3 million", CS-2 costs "several" million. A single Nvidia H100 costs somewhere around $30,000 each, so a GPU server with every slot populated costs about $300,000.
https://www.servethehome.com/graphcore-celebrates-a-stunning...
Not sure about the H100, but it seems to be more supply constrained (hence pricier) atm.
Now, the real question is how many HGX nodes "equals" a single CS2 node. The math here is extremely fuzzy, as the benefit to such extreme node consolidation depends on the workload, and the CS-2 takes up less space, but the HGX cluster will have more directly accessible RAM and better turnkey support for stuff since its Nvidia.
Earlier quoted context omitted.
Serious question: is it typical to describe client-side computing as "on the edge"? I thought running something on the edge referred to running it in close network proximity to the user, rather than users having control and running things themselves.
I believe this has been extended to mean "on device", which is interesting. See Gerganov's article on Github [0]. I wrote about this here [1] where I made a contrast between the core and the edge. I think the term maps well to this meaning. What I find more interesting is that in the classic "close network proximity", some parts of the world may not have benefited as much from that trend since the closest nodes of a…
I don't like the connotations this carries. This is almost openly talking about reaching all the way into peoples' hardware to run your software, for your benefit, on them, without their knowledge, consent or control...
This type of article (or press release, or whatever you want to call it) is exactly what makes the future so interesting. The cat is out of the bag, the genie is out of the bottle, the confetti has left the cannon[0]. It's tempting to see a world dominated by Google Bard, ChatGPT, Bing Search, etc. And no doubt, they will be huge players, with services that are far more powerful than anything that can be run on the e…
Serious question: is it typical to describe client-side computing as "on the edge"? I thought running something on the edge referred to running it in close network proximity to the user, rather than users having control and running things themselves.
Somewhat; its consistent with, e.g., Google’s “Edge TPU” designation for its client-side neural processors.
> I thought running something on the edge referred to running it in close network proximity to the user
Typically, but on the client device is the limit-case of “close network proximity to the user”, so the use is consistent.
I wish rather than stopping training early they would have run more data through a small model so we could have something more competitive with LLaMA 7B.
Here are the zero-shot accuracy numbers posted in the Huggingface evaluations for Cerebras-GPT 13B vs. the results of LLaMa 13B in their paper: Model BoolQ PIQA SIQA HellaSwag WinoGrande ARC-e ARC-c OBQA LLaMa 13B 78.1 80.1 50.4 79.2 73 74.8 52.7 56.4 Cerebras-GPT 13B - 76.6 - 51.3 64.6 71.4 36.7 28.6
I guess it's something. It still goes to show how far open models are behind the proprietary SOTA.
> It would be interesting to know why you chose those FLOPS targets, unfortunately it looks like the models are quite under pre-trained (260B tokens for 13B model)
> We chose to train these models to 20 tokens per param to fit a scaling law to the Pile data set. These models are optimal for a fixed compute budget, not necessarily "best for use". If you had a fixed parameter budget (e.g., because you wanted to fit models on certain hardware) you would train on more tokens. We do that for our customers that seek that performance and want to get LLaMA-like quality with a commercial license
Which is the point made elsewhere in these comments, e.g. https://news.ycombinator.com/item?id=35344192, and also usefully shows how open Cerebras are. They're pretty open, but not as much as they would be if they were optimising for filling in other companies' moats.
I wonder what led to such a gap between llama 7b and Cerebras 13b. I hope they discuss it in the paper.
From the Cerebras blog post: "Trained using the Chinchilla formula, these models provide the highest accuracy for a given compute budget."
From the LLaMA paper: "The focus of this work is to train a series of language models that achieve the best possible performance at various inference budgets, by training on more tokens than what is typically used."
Earlier quoted context omitted.
Yes, yes, and yes. I'm waiting for an actually open AI that can run on the edge, purely on commodity hardware like our laptops and phones - it's inevitable. I imagine this "cat out of the bag" situation, the democratization and commodification of powerful technology accessible and affordable to the public, is similar to what's happening with single-board computers and microcontrollers like Raspberry Pi, Arduino, ESP3…
After using ChatGPT 4 extensively for a few days, I think we're probably only a few years away from the first generation of truly conversational assistants ala Jarvis in Iron Man. Between LangChain and existing voice recognition software, we've already 95% of the way there, it just needs to be packaged up into a UI/UX that makes sense. These local models are absolutely critical for that to happen though. I'm hitting…