Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

anandtech.com

Post reply on HN
Print view View original

11–20 of 46 posts

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#11

Post by colechristensen »

Uptrenda wrote: post #6

What is up with the price? 40k for only 2 x 48 cores and a mediocre 2 ghz clock rate. Modern processors are way more efficient than older processors but I simply can't imagine the efficiency adds up to (simplistically) $416 per core... Very interesting memory bus though... 1 TB / s? That is cool, but I would still much rather get a crap load more cores at a reasonable price then be able to send around data that effic…

This is a scaled down version of a single unit that’s designed to go into supercomputers by the thousand, where the cost of the whole system is hundreds of millions of dollars, consuming megawatts of power.

You would probably buy one of these to facilitate testing and development of codes to run on a big one.

No doubt the memory vs cpu balance is correct for what these computers are doing.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#12

Post by reitzensteinm »

FeepingCreature wrote: post #7
reitzensteinm wrote: post #5

Earlier quoted context omitted.

It has 1 TB/s memory bandwidth via HBM2, and 512 bit vector units. A better comparison would probably be to Xeon Phi (not that it's lighting the world on fire exactly).

Ah fair, I think the 3990x only gets 100GB/s at quad DDR4 3200, if I'm interpreting the Google results right. That could be a killer. And there's no AVX512 support either (though it does have -256). Would be good to have some actual benchmarks to compare.

I'd also not focus too much on single node value for money; these machines come with Infiniband and are likely intended to be evaluation units for people that are interested in building a large cluster.

Fugaku cost $1 billion, with 159k nodes. That's only ~$6k per node, and that's if you assume the entire budget is allocated to the compute nodes, which it wouldn't nearly be. Wikipedia lists it as total program cost; so in addition to the facility, cooling and other equipment such as networking, it may well include years worth of operating expenses, staff, etc.

I believe the best way to view the price tag is: low enough that it's no barrier to anyone that's interested in building even a moderate cluster, but high enough to keep away anyone that's just toying around.

That way if somebody buys one of these machines and calls you up to ask a question, you're not wasting your time to treat it as a step in a longer sales process (meaning you let them talk to engineers, respond in depth etc).

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#13

Post by colechristensen »

FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

Support and software, the case, the motherboards, power supplies, memory, the fact that the threadripper uses twice as much power, reliability testing: in a real supercomputer you’ll be buying thousands of them and dealing with failures gets expensive for everyone... and on and on

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#14

Post by sdgjhfdlkgjhdf »

FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

I think that a useful way of seeing this is to compare car engines:

From wikipedia, a Ford Flex Limited can have a V6 turbocharged 3.6L engine (max torque 350 lb-ft), while a Formula One has a V6 turbocharged with only 1.6L (max torque 214 lb-ft).

The Formula One engine has lower specs, and cost orders of magnitude more.

I think that the answer is that this comparison doesn't make much sense, when the use cases and the target are so different.

Edit: it is funny to see all the replies to my comment mirroring the replies to the parent comment: there is a lot more to the comparison than just cherry-picking a few spec point.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#15

Post by outsomnia »

sdgjhfdlkgjhdf wrote: post #14
FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

I think that a useful way of seeing this is to compare car engines: From wikipedia, a Ford Flex Limited can have a V6 turbocharged 3.6L engine (max torque 350 lb-ft), while a Formula One has a V6 turbocharged with only 1.6L (max torque 214 lb-ft). The Formula One engine has lower specs, and cost orders of magnitude more. I think that the answer is that this comparison doesn't make much sense, when the use cases and t…

They are just aarch64 cores like in your phone, not a "formula 1 engine". I think it's a really tough sell.

For around the same money you can buy 512 cores of modern AMD x86_64 (4 x 128 core, ~$5000 ea) AND 4 x rtx8000 gpu, 18,000 CUDA compute cores (also ~$5000 ea)

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#16

Post by formerly_proven »

nottorp wrote: post #9
Uptrenda wrote: post #6

What is up with the price? 40k for only 2 x 48 cores and a mediocre 2 ghz clock rate. Modern processors are way more efficient than older processors but I simply can't imagine the efficiency adds up to (simplistically) $416 per core... Very interesting memory bus though... 1 TB / s? That is cool, but I would still much rather get a crap load more cores at a reasonable price then be able to send around data that effic…

If you need that kind of memory bandwidth you'll probably know already, and maybe this will even look cheap. HBM not only has lotsa bandwidth(tm) but also much better latency?

Latency in conventional DDRx memory is limited by the DRAM array itself, which doesn't change with the interface (DDR, GDDR, HBM, ...). Essentially, with regular DRAM, you cannot do better than the contemporary low-latency DDRx memory. You might chose to increase latency to increase throughput, though.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#17

Post by mhh__ »

sdgjhfdlkgjhdf wrote: post #14
FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

I think that a useful way of seeing this is to compare car engines: From wikipedia, a Ford Flex Limited can have a V6 turbocharged 3.6L engine (max torque 350 lb-ft), while a Formula One has a V6 turbocharged with only 1.6L (max torque 214 lb-ft). The Formula One engine has lower specs, and cost orders of magnitude more. I think that the answer is that this comparison doesn't make much sense, when the use cases and t…

Keep in mind that the F1 engine can pump out 1000HP almost instantly and is more than 50% thermally efficient (Approximately double a road car)

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#18

Post by formerly_proven »

sdgjhfdlkgjhdf wrote: post #14
FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

I think that a useful way of seeing this is to compare car engines: From wikipedia, a Ford Flex Limited can have a V6 turbocharged 3.6L engine (max torque 350 lb-ft), while a Formula One has a V6 turbocharged with only 1.6L (max torque 214 lb-ft). The Formula One engine has lower specs, and cost orders of magnitude more. I think that the answer is that this comparison doesn't make much sense, when the use cases and t…

The Formula One engine has about 3-4x the crankshaft power of that Ford V6. The F1 engine has higher specs all around except weight and torque; the latter is irrelevant since gears exist.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#19

Post by marktangotango »

Uptrenda wrote: post #6

What is up with the price? 40k for only 2 x 48 cores and a mediocre 2 ghz clock rate. Modern processors are way more efficient than older processors but I simply can't imagine the efficiency adds up to (simplistically) $416 per core... Very interesting memory bus though... 1 TB / s? That is cool, but I would still much rather get a crap load more cores at a reasonable price then be able to send around data that effic…

What’s the SHA256 hash rate and power consumption? Only partially joking btw!

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#20

Post by dman »

outsomnia wrote: post #15
sdgjhfdlkgjhdf wrote: post #14

Earlier quoted context omitted.

I think that a useful way of seeing this is to compare car engines: From wikipedia, a Ford Flex Limited can have a V6 turbocharged 3.6L engine (max torque 350 lb-ft), while a Formula One has a V6 turbocharged with only 1.6L (max torque 214 lb-ft). The Formula One engine has lower specs, and cost orders of magnitude more. I think that the answer is that this comparison doesn't make much sense, when the use cases and t…

They are just aarch64 cores like in your phone, not a "formula 1 engine". I think it's a really tough sell. For around the same money you can buy 512 cores of modern AMD x86_64 (4 x 128 core, ~$5000 ea) AND 4 x rtx8000 gpu, 18,000 CUDA compute cores (also ~$5000 ea)

They are the only cores you can buy right now with the SVE extensions - so not exactly equivalent too the phone cores.
Post reply on HN