Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cray versus Raspberry Pi

aardvark.co.nz

Post reply on HN
Print view View original

91–100 of 136 posts

Re: Cray versus Raspberry Pi

#91

Post by alganet »

rahen wrote: post #89
adwn wrote: post #85

Earlier quoted context omitted.

> EXO Labs showed otherwise by getting a 300K-parameter LLM to run on a Pentium II with only 128 MB of RAM at about 50 tokens per second 50 token/s is completely useless if the tokens themselves are useless. Just look at the "story" generated by the model presented in your link: Each individual sentence is somewhat grammatically correct, but they have next to nothing to do with each other, they make absolutely no sen…

You're missing the point. No one is claiming that a 300K-param model on a Pentium II matches GPT-4. The point is that it works: it parses input, generates plausible syntax, and does so using algorithms and compute budgets that were entirely feasible decades ago. The claim is that we could have explored and deployed narrow AI use cases decades earlier, had the conceptual focus been there. Even at that small scale, you…

> we weren’t held back by hardware

Llama 3 8B took 1.3M hours to train in a H100-80GB.

Of course, it didn't took 1.3M hours (~150 years). So, many machines with 80GB were used.

Let's do some napkin math. 150 machines with a total of 12TB VRAM for a year.

So, what would be needed to train a 300K parameter model that runs on 128MB RAM? Definitely more, much more than 128MB RAM.

Llama 3 runs on 16GB VRAM. Let's imagine that's our Pentium II of today. You need at least 750 times what is needed to run it in order to train it. So, you would have needed ~100GB RAM back then, running for a full year, to get that 300K model.

How many computers with 100GB+ RAM do you think existed in 1997?

Also, I only did RAM. You also need raw processing power and massive amounts of training data.

Re: Cray versus Raspberry Pi

#92

Post by lttlrck »

ForOldHack wrote: post #57

Adjust the price of the Cray-1, for inflation, but not the power, for Moore's law? Need I get my napkin out for a few calculations? or do we just FORGET MOORE'S LAW ( that is mention no less that 4 times, without quantification? Cray-1 (1976 ). RPi ( 2012 ). 37 years of elapsed time. 24. 2/3 elapsed generations. 26,509,000 times increase in power. Cray 1 160Mf. In a 26M times faster, would yield 4,241Gf ( 4.2Pf) , wh…

However Moores Law refers to the number of transistors. Not FLOPS.

Re: Cray versus Raspberry Pi

#93

Post by rahen »

alganet wrote: post #91
rahen wrote: post #89

Earlier quoted context omitted.

You're missing the point. No one is claiming that a 300K-param model on a Pentium II matches GPT-4. The point is that it works: it parses input, generates plausible syntax, and does so using algorithms and compute budgets that were entirely feasible decades ago. The claim is that we could have explored and deployed narrow AI use cases decades earlier, had the conceptual focus been there. Even at that small scale, you…

> we weren’t held back by hardware Llama 3 8B took 1.3M hours to train in a H100-80GB. Of course, it didn't took 1.3M hours (~150 years). So, many machines with 80GB were used. Let's do some napkin math. 150 machines with a total of 12TB VRAM for a year. So, what would be needed to train a 300K parameter model that runs on 128MB RAM? Definitely more, much more than 128MB RAM. Llama 3 runs on 16GB VRAM. Let's imagine…

You’re basically arguing that because A380s need millions of liters of fuel and a 4km runway, the Wright Flyer was impossible in 1903. That logic just doesn’t hold. Different goals, different scales, different assumptions. The 300K model shows that even in the 80s, it was both possible and sufficient for narrow but genuinely useful tasks.

We simply weren’t looking, blinded by symbolic programming and expert systems. This could have been a wake-up call, steering AI research in a completely different direction and accelerating progress by decades. That’s the whole point.

Re: Cray versus Raspberry Pi

#94

Post by darkwater »

usrnm wrote: post #78
dale_huevo wrote: post #77

> the Cray had about 160MFLOPS of raw processing power; the Pi has... up to 30GFLOPS. Yes... that's giga FLOPS. This makes it almost 200 times faster than the Cray. Imagine traveling back to 1977 and explaining to someone that in 2025 we've allocated all that extra computing power to processing javascript bundles and other assorted webshit.

That actually wouldn't be so bad, but in reality the number one usecase for raspberry pi is blinking leds for some time and collecting dust afterwards

Still a better user than crunching Javascript to show you ads and track you around.

Re: Cray versus Raspberry Pi

#95

Post by alganet »

rahen wrote: post #93
alganet wrote: post #91

Earlier quoted context omitted.

> we weren’t held back by hardware Llama 3 8B took 1.3M hours to train in a H100-80GB. Of course, it didn't took 1.3M hours (~150 years). So, many machines with 80GB were used. Let's do some napkin math. 150 machines with a total of 12TB VRAM for a year. So, what would be needed to train a 300K parameter model that runs on 128MB RAM? Definitely more, much more than 128MB RAM. Llama 3 runs on 16GB VRAM. Let's imagine…

You’re basically arguing that because A380s need millions of liters of fuel and a 4km runway, the Wright Flyer was impossible in 1903. That logic just doesn’t hold. Different goals, different scales, different assumptions. The 300K model shows that even in the 80s, it was both possible and sufficient for narrow but genuinely useful tasks. We simply weren’t looking, blinded by symbolic programming and expert systems.…

"I mean, today we can do jet engines in garage shops. Why would they needed a catapult system? They could have used this simple jet engine. Look, here is the proof, there's a YouTuber that did a small tiny jet engine in his garage. They were held back by ideas, not aerodynamics and tooling precision."

See how silly it is?

Now, focus on the simple question. How would you train the 300K model in 1997? To run it, you someone to train it first.

Re: Cray versus Raspberry Pi

#96

Post by rahen »

alganet wrote: post #95
rahen wrote: post #93

Earlier quoted context omitted.

You’re basically arguing that because A380s need millions of liters of fuel and a 4km runway, the Wright Flyer was impossible in 1903. That logic just doesn’t hold. Different goals, different scales, different assumptions. The 300K model shows that even in the 80s, it was both possible and sufficient for narrow but genuinely useful tasks. We simply weren’t looking, blinded by symbolic programming and expert systems.…

"I mean, today we can do jet engines in garage shops. Why would they needed a catapult system? They could have used this simple jet engine. Look, here is the proof, there's a YouTuber that did a small tiny jet engine in his garage. They were held back by ideas, not aerodynamics and tooling precision." See how silly it is? Now, focus on the simple question. How would you train the 300K model in 1997? To run it, you so…

Reductio ad absurdum. A 300K-param model was small enough to be trained offline, on curated datasets, with CPUs and RAM capacities that absolutely existed at the time, especially in research centers.

Backprop was known. Data was available. Narrow tasks (completion, summarization, categorization) were relevant. The model that runs on a Pentium II could have been trained on a Cray, or across time on any reasonably powerful 90s workstation. That’s not fantasy, LeNet 5 with its 65K weight was trained on a mere Sun station in the early 90s.

The limiting factor wasn’t compute, it was the conceptual framing as well as the datasets. No one seriously tried, because the field was dominated by symbolic logic and rule-based AI. That’s the core of the argument.

Re: Cray versus Raspberry Pi

#97

Post by username223 »

_fat_santa wrote: post #33

Reading this I wonder, say we did have a time machine and were somehow able to give scientists back in the day access to an RPI5. What sort of crazy experiments would that have spawned? I'm sure when the Cray 1 came out, access to it must have been very restricted and there must have been hoards of scientists clamoring to run their experiments and computations on it. What would have happened if we gave every one of t…

> What sort of crazy experiments would that have spawned?

Scientists then (at least a lot of them) knew what they wanted to do, and it required faster computers rather than more of them. A lot of that Cray power at the national labs was doing fluid simulation (i.e. nuclear explosions), and with the computers they had in the 80s, it was done in one or two dimensions, relying on symmetry. Going from n^2 to n^3 grid cells was the obvious next step, but took a lot more memory and CPU speed.

Re: Cray versus Raspberry Pi

#98

Post by monocasa »

nottorp wrote: post #80
dahart wrote: post #49

My former boss (Steve Parker, RIP) shared a story of Turner Whitted making predictions about how much compute would be needed to achieve real-time ray tracing, some time around when his seminal paper was published (~1980). As the story goes, Turner went through some calculations and came to the conclusion that it’d take 1 Cray per pixel. Because of the space each Cray takes, they’d be too far apart and he thought the…

But the Cray had a general purpose CPU while the GPUs have specialized hardware. Not exactly apples to apples.

The main part of the Cray was a compute offload engine that asynchronously executed job lists submitted by front end general purpose computers that ran OSes like Unix.

It was actually pretty close to the model of a GPU.

Re: Cray versus Raspberry Pi

#99

Post by username223 »

That was a weird turn to AI at the end, but otherwise an interesting reflection. I'm a little too young to have grown up in the era of the Cray-1, but even in the early 90s, processors ran at 90 MHz and hard drives cost $1 per megabyte. Back when personal computers ran at single-digit megahertz and had kilobytes of RAM, a Cray was mind-blowing.

The exciting part back then was that, while computers were never "good enough," they were getting noticeably better every few months. If you were in the market for a computer, you knew you could get a noticeably better one for the same price if you just waited a little while. The next model was exciting, because it was tangibly better. At some point personal computers became "good enough" for most people. Other than compensating for creeping software bloat, there hasn't been much reason for most people to be excited about new computers in a decade or more.

Re: Cray versus Raspberry Pi

#100

Post by alganet »

rahen wrote: post #96
alganet wrote: post #95

Earlier quoted context omitted.

"I mean, today we can do jet engines in garage shops. Why would they needed a catapult system? They could have used this simple jet engine. Look, here is the proof, there's a YouTuber that did a small tiny jet engine in his garage. They were held back by ideas, not aerodynamics and tooling precision." See how silly it is? Now, focus on the simple question. How would you train the 300K model in 1997? To run it, you so…

Reductio ad absurdum. A 300K-param model was small enough to be trained offline, on curated datasets, with CPUs and RAM capacities that absolutely existed at the time, especially in research centers. Backprop was known. Data was available. Narrow tasks (completion, summarization, categorization) were relevant. The model that runs on a Pentium II could have been trained on a Cray, or across time on any reasonably powe…

> Reductio ad absurdum.

My dude, you came up with the Wright brothers comparison, not me. If you don't like fallacies, don't use them.

> on any reasonably powerful 90s workstation

https://hal.science/hal-03926082/document

Quoting the paper now:

> In 1989 a recognizer as complex as LeNet-5 would have required several weeks’ training and more data than were available and was therefore not even considered.

Their own words seem to match my assessment.

Training time and data availability determined how much this whole thing could advance, and researchers were aware of those limits.

Post reply on HN