Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GigaGPT: GPT-3 sized models in 565 lines of code

cerebras.net

Post reply on HN
Print view View original

31–40 of 70 posts

Re: GigaGPT: GPT-3 sized models in 565 lines of code

#31

Post by cbsmith »

TeMPOraL wrote: post #14
icyfox wrote: post #7

For those hearing about Cerebras for the first time, they make a chipset that's similar to a GPU in matrix multiplication speed but way bigger (a whole wafer) so it can fit more transistors and memory onto one chip. They achieve this small LOC count because they don't need to shard across multiple devices / backprop consolidate on a central CPU / etc. These tricks are usually what blows up a project from a single arc…

This does however feel a bit like that "big data" phenomenon, where most companies deploy ridiculously overcomplicated distributed data clusters, where their actual problems could be handled much more simply, cheaply and efficiently, by a single server with a lots of RAM and a solution somewhere on the spectrum between "bunch of UNIX pipes with standard UNIX text processing tools" and "tuned PostgreSQL" / "tuned in-m…

In fairness, it's more a case of companies incorrectly identifying their problem as a "big data" problem, usually due to ego or resume padding. If you genuinely have a "big data" problem, you probably do need distributed data clusters.

Re: GigaGPT: GPT-3 sized models in 565 lines of code

#32

Post by convery »

101008 wrote: post #24

Would it be ever possible to run a GPT-{n}, n>3, similar model in a home computer wihtout GPU? I have a "good" laptop with 32GB, good processor, but no GPU (I was never interested in gaming, crypto or ML), but I found GPT very useful and I'd prefer to run a local version instead of keep feeding OpenAI.

Most laptops these days have a pretty sizable GPU on the same chip. IIRC Triton makes proper use of the Intel graphics while AMDs equivalents work well with OpenCL out of the box. Apple's M1-3 architecture saw some major speedups on llama.cpp etc. as well. Worth noting is that some may need special drivers, my Xeons from 2010 has support for executing OpenCL but needed extra drivers; no comment on modern processors.

Re: GigaGPT: GPT-3 sized models in 565 lines of code

#34

Post by kristjansson »

TeMPOraL wrote: post #23
kristjansson wrote: post #20

Earlier quoted context omitted.

You can hide a whole lot of essential complexity in a hardware layer. However, the very next question a researcher will ask once a model fits on one device is “can I make it twice as fast/big if I use two?”

I'd think that such researcher would've already heard of adages like "you can't get nine women to give birth in one month", or "where there's six cooks, there's nothing to eat". Or more directly, perhaps one should ask such researcher, "if your team was to double in head count, would you do this project twice as fast?".

8 GPUs do a pretty bang-up job of doing 8 months of compute in a month :).

I think the broader point is that the last x0 years of ML research show that more compute is better, both for iteration speed and for resulting performance. Distribution is just the natural outgrowth of that imperative once it reaches the limit of a single device/node. If Cerebras can address models at today's scale on one device, the immediate next step is "what can N of these devices do together to build models at tomorrow's scale"

[0]: https://www.cerebras.net/condor-galaxy-1

Re: GigaGPT: GPT-3 sized models in 565 lines of code

#35

Post by filterfiber »

I don't understand why they're comparing the parameter sizes to lines of code.

AFAIK you can just increase the layer parameters of a 1B model to whatever you want? Like, the difference between a 1B and 175B model can be just changing a few numbers, and not adding any LOC at all?

LOC has never been a limitation for large models, it's been the compute+training data required.

Most of the LOC is spent on optimization, and they don't address MoE or anything fancy like that?

Re: GigaGPT: GPT-3 sized models in 565 lines of code

#36

Post by cs702 »

icyfox wrote: post #18
TeMPOraL wrote: post #14

Earlier quoted context omitted.

This does however feel a bit like that "big data" phenomenon, where most companies deploy ridiculously overcomplicated distributed data clusters, where their actual problems could be handled much more simply, cheaply and efficiently, by a single server with a lots of RAM and a solution somewhere on the spectrum between "bunch of UNIX pipes with standard UNIX text processing tools" and "tuned PostgreSQL" / "tuned in-m…

I'm a big believer in the approach you're laying out too. Bugs are much easier to diagnose, crash reporting is more straightforward, and you don't need augmented services to consolidate everything at the output layer. That said - I've been predicting a shift back to simple architectures for awhile now and they haven't really come to pass. Maybe there's too much pressure or financial incentives for increasing complexi…

The other thing to keep in mind is that Transformers may well be end up being supplanted by more efficient alternatives with different hardware requirements.

For instance, right now there's a new crop of "linear RNNs" (RWKV, Mamba, retnet, etc.) claiming to be as good as Transformers for language modeling but with two advantages: their compute cost is O(n) instead of O(n²), and they don't need to keep past context in memory.

I don't know if these linear RNNs will actually supplant Transformers, but I do think hardware requirements are likely to change over time.

Re: GigaGPT: GPT-3 sized models in 565 lines of code

#37

Post by happytiger »

icyfox wrote: post #18
TeMPOraL wrote: post #14

Earlier quoted context omitted.

This does however feel a bit like that "big data" phenomenon, where most companies deploy ridiculously overcomplicated distributed data clusters, where their actual problems could be handled much more simply, cheaply and efficiently, by a single server with a lots of RAM and a solution somewhere on the spectrum between "bunch of UNIX pipes with standard UNIX text processing tools" and "tuned PostgreSQL" / "tuned in-m…

I'm a big believer in the approach you're laying out too. Bugs are much easier to diagnose, crash reporting is more straightforward, and you don't need augmented services to consolidate everything at the output layer. That said - I've been predicting a shift back to simple architectures for awhile now and they haven't really come to pass. Maybe there's too much pressure or financial incentives for increasing complexi…

Which architecture more closely resembles the human brain?

Re: GigaGPT: GPT-3 sized models in 565 lines of code

#39

Post by jwan584 »

filterfiber wrote: post #35

I don't understand why they're comparing the parameter sizes to lines of code. AFAIK you can just increase the layer parameters of a 1B model to whatever you want? Like, the difference between a 1B and 175B model can be just changing a few numbers, and not adding any LOC at all? LOC has never been a limitation for large models, it's been the compute+training data required. Most of the LOC is spent on optimization, an…

when you go from 1B to 175B, the model no longer fits in memory. so in practice you have to re-factor the model using tensor/pipeline parallelism. that's why it goes from 600 to 20K LOC.

Re: GigaGPT: GPT-3 sized models in 565 lines of code

#40

Post by 101008 »

whynotmaybe wrote: post #28
101008 wrote: post #24

Would it be ever possible to run a GPT-{n}, n>3, similar model in a home computer wihtout GPU? I have a "good" laptop with 32GB, good processor, but no GPU (I was never interested in gaming, crypto or ML), but I found GPT very useful and I'd prefer to run a local version instead of keep feeding OpenAI.

Have you checked Gpt4all or Faraday.dev ?

I haven't but I will - thanks!!
Post reply on HN