Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cray versus Raspberry Pi

aardvark.co.nz

Post reply on HN
Print view View original

61–70 of 136 posts

Re: Cray versus Raspberry Pi

#61

Post by dahart »

magicalhippo wrote: post #52
dahart wrote: post #49

My former boss (Steve Parker, RIP) shared a story of Turner Whitted making predictions about how much compute would be needed to achieve real-time ray tracing, some time around when his seminal paper was published (~1980). As the story goes, Turner went through some calculations and came to the conclusion that it’d take 1 Cray per pixel. Because of the space each Cray takes, they’d be too far apart and he thought the…

> 1 Cray per pixel would be 49 Tera flops. A 4080 GPU has just shy of 50 Tflops peak performance Interesting, wonder how it compares in terms of transistors. How many transistors combined did one Cray have in compute and cache chips?

The Wikipedia article says the Cray-1 has 200k gates. I assume that would mean something slightly north of 2x the number of transistors? https://en.wikipedia.org/wiki/Cray-1#Description

200k * 300k Cray-1s would be 60B gates, whereas the 4080 actually has 46B transistors. Seems like we’re totally in the right ballpark.

Re: Cray versus Raspberry Pi

#63

Post by jwr »

What I find somewhat puzzling is that these machines were used for the "really big problems". We used supercomputers for weather forecasting, finite element simulations, molecular modeling. And we were getting results.

I don't feel we are getting results that are thousands of times better today.

Re: Cray versus Raspberry Pi

#64

Post by hattmall »

dahart wrote: post #49

My former boss (Steve Parker, RIP) shared a story of Turner Whitted making predictions about how much compute would be needed to achieve real-time ray tracing, some time around when his seminal paper was published (~1980). As the story goes, Turner went through some calculations and came to the conclusion that it’d take 1 Cray per pixel. Because of the space each Cray takes, they’d be too far apart and he thought the…

Nice, but the ~40 year latency is kind 0f high.

Re: Cray versus Raspberry Pi

#65

Post by adwn »

rahen wrote: post #47
delichon wrote: post #3

> but then again if you'd showed me an RPi5 back in 1977 I would have said "nah, impossible" so who knows? I was reading lots of scifi in 1977, so I may have tried to talk to the pi like Scotty trying to talk to the mouse in Star Trek IV. And since you can run an LLM and text to speech on an RPi5, it might have answered.

No need for an RPi 5. Back in 1982, a dual or quad-CPU X-MP could have run a small LLM, say, with 200–300K weights, without trouble. The Crays were, ironically, very well suited for neural networks, we just didn’t know it yet. Such an LLM could have handled grammar and code autocompletion, basic linting, or documentation queries and summarization. By the late 80s, a Y-MP might even have been enough to support a small…

> a small LLM, say, with 200–300K weights

A "small Large Language Model", you say? So a "Language Model"? ;-)

> Such an LLM could have handled grammar and code autocompletion, basic linting, or documentation queries and summarization.

No, not even close. You're off by 3 orders of magnitude if you want even the most basic text understanding, 4 OOM if you want anything slightly more complex (like code autocompletion), and 5–6 OOM for good speech recognition and generation. Hardware was very much a limiting factor.

Re: Cray versus Raspberry Pi

#66

Post by motorest »

jwr wrote: post #63

What I find somewhat puzzling is that these machines were used for the "really big problems". We used supercomputers for weather forecasting, finite element simulations, molecular modeling. And we were getting results. I don't feel we are getting results that are thousands of times better today.

> I don't feel we are getting results that are thousands of times better today.

You are getting results that are way better than thousands of times. You just aren't aware where they are showing up.

To give you a glimpse, the same modelling problems which a couple of decades ago tool days to come up with a crude solution are now being executed within a loop in optimization problems.

You are also seeing multiphysics and coupling problems showing up in mundane applications. We're talking about problems that augment the same modelling problems that a couple of decades ago tool days to solve with double or triple the degrees of freedom.

Without the availability of these supercomputers the size of credit cards, the whole field of computer-aided engineering would not exist.

Also, to boot, there are indeed diminished returns. Increasing computational resources unblocks constraints such as being able to use doubles instead of floats. This means that lowering numerical errors in 3 or 4 decimal places comes for free at the expense of taking around 4 times longer to solve the same problem.

To top things off, do you think the results of two decades ago were possible without employing a great deal of simplifications and crude approximations? As legend has it, the F117 Nighthawk got it's design due to the computational limits of the time. Since then, stealth planes became more performant and with a smoother design. That's what you get when your computational resources are a thousands times better.

Re: Cray versus Raspberry Pi

#67

Post by tsoukase »

Mountain_Skies wrote: post #5
delichon wrote: post #3

> but then again if you'd showed me an RPi5 back in 1977 I would have said "nah, impossible" so who knows? I was reading lots of scifi in 1977, so I may have tried to talk to the pi like Scotty trying to talk to the mouse in Star Trek IV. And since you can run an LLM and text to speech on an RPi5, it might have answered.

Someday real soon, kids being shown episodes of 'Knight Rider' by their grandparents won't understand why a talking car was so futuristic.

I grew up watching Kitt and when I watched it again a few days ago, I didn't feel anything. Much less my kids.

Re: Cray versus Raspberry Pi

#68

Post by ziofill »

It is a frequent fantasy of mine to bring tech back to historical figures, like to show my phone to Galileo or to take Leonardo da Vinci for a ride in my car. But I guess you don't need to go that far to blow minds.

Re: Cray versus Raspberry Pi

#69

Post by Aardwolf »

mikewarot wrote: post #43
_fat_santa wrote: post #33

Reading this I wonder, say we did have a time machine and were somehow able to give scientists back in the day access to an RPI5. What sort of crazy experiments would that have spawned? I'm sure when the Cray 1 came out, access to it must have been very restricted and there must have been hoards of scientists clamoring to run their experiments and computations on it. What would have happened if we gave every one of t…

First of all, how would they talk to it? You'd have to give them an RPI5 with serial console enabled, and strict instructions not to exceed the 3.3 volt limits of the I/O. Now it's reasonable that you could generate NTSC video out of it, so they could see on the screen any output. When you then explained it was just bit-banging said NTSC output, they'd be amazed even more.

Give it also an hdmi screen and usb keyboard, what more do you need to type code and see the result

Re: Cray versus Raspberry Pi

#70

Post by nereye »

bsoles wrote: post #60
Mountain_Skies wrote: post #10

Earlier quoted context omitted.

Not really. My 1983 Datsun would talk, but it couldn't converse. Alexa and Siri couldn't hold a conversation anywhere near the level KITT did. There's a big difference. With LLMs, we're getting close.

Commodore 64 had text to speech in late 80s. Also, my friend's father in the 80s was the driver of a French Consulate's member in Turkey. His car (a Renault) had speech functionality.

Early 80s (1982), according to Wikipedia:

https://en.m.wikipedia.org/wiki/Software_Automatic_Mouth

Post reply on HN