Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

anandtech.com

Post reply on HN
Print view View original

21–30 of 46 posts

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#21

Post by microcolonel »

formerly_proven wrote: post #16
nottorp wrote: post #9

Earlier quoted context omitted.

If you need that kind of memory bandwidth you'll probably know already, and maybe this will even look cheap. HBM not only has lotsa bandwidth(tm) but also much better latency?

Latency in conventional DDRx memory is limited by the DRAM array itself, which doesn't change with the interface (DDR, GDDR, HBM, ...). Essentially, with regular DRAM, you cannot do better than the contemporary low-latency DDRx memory. You might chose to increase latency to increase throughput, though.

> Essentially, with regular DRAM, you cannot do better than the contemporary low-latency DDRx memory. You might chose to increase latency to increase throughput, though.

My understanding is that, given how memory systems work right now, typically it's the opposite: increasing throughput decreases latency.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#23

Post by tyingq »

FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

I don't think anyone buys a 2 node HPC system for production. Note that you can fit 6 more nodes into this 2U system. 288 cores in 2U with infiniband, etc.

This is likely the development setup for someone that also buys a fully populated rack for production. So it's probably not meant to be a great $$/core story at this small scale.

Though, your point that the AMD EPYC is probably pressuring these ARM HPC setups is fair.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#24

Post by moonbug »

FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

if you have to ask it's not for you. this is for people who need a PoC or reference for a larger HPC system. the pricing is largely irrelevant.

as to the system itself, most of the value in this processor is its memory bandwidth, derived from HBM integration. no other cpu system comes close.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#25

Post by mbreese »

Uptrenda wrote: post #6

What is up with the price? 40k for only 2 x 48 cores and a mediocre 2 ghz clock rate. Modern processors are way more efficient than older processors but I simply can't imagine the efficiency adds up to (simplistically) $416 per core... Very interesting memory bus though... 1 TB / s? That is cool, but I would still much rather get a crap load more cores at a reasonable price then be able to send around data that effic…

See, and I had the opposite opinion. I work with HPC all the time (computational biology) and have built (small) clusters. When I saw the price I thought — yep, that make sense for a novel (and core heavy) HPC node. It’s on the high end, but not an outrageous price. Normally with something so new, you never actually see a price (if you have to ask...).

Especially when you consider that this is a new HPC architecture, that’s not an unreasonable price. As others have mentioned, this is a SKU you’re buying to evaluate a significantly larger purchase.

There isn’t exactly a lot of pricing pressure on the processors. Top end HPC always has the biggest margins because this is a market where absolute speed tends to trump all other factors, including price. A better price comparison might be an IBM Power HPC node — another fairly novel architecture built for raw speed and HPC.

Plus, it is entirely possible that the cost difference of the node (compared to x86) is completely covered by the power savings. HPC is very power hungry. So a significant savings there could make up for an increased node cost.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#26

Post by fock »

q3k wrote: post #8
FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

You're comparing the price of a consumer CPU to an entire rackable unit. There's a whole bunch of more stuff that goes into making a good server than just stuffing a good CPU into a beige box.

then look at those 2U, 4x2 socket supermicros and think again. 512 epyc cores and 4TB of memory + Infiniband sell for USD 100k.

but you're probably right that noone buys 2 of these at that price. Probably only japanese companies proud of their country or whatever.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#27

Post by numpad0 »

outsomnia wrote: post #15
sdgjhfdlkgjhdf wrote: post #14

Earlier quoted context omitted.

I think that a useful way of seeing this is to compare car engines: From wikipedia, a Ford Flex Limited can have a V6 turbocharged 3.6L engine (max torque 350 lb-ft), while a Formula One has a V6 turbocharged with only 1.6L (max torque 214 lb-ft). The Formula One engine has lower specs, and cost orders of magnitude more. I think that the answer is that this comparison doesn't make much sense, when the use cases and t…

They are just aarch64 cores like in your phone, not a "formula 1 engine". I think it's a really tough sell. For around the same money you can buy 512 cores of modern AMD x86_64 (4 x 128 core, ~$5000 ea) AND 4 x rtx8000 gpu, 18,000 CUDA compute cores (also ~$5000 ea)

It’s actually Fujitsu’s former SPARC core modified to take AArch64 instructions, or at least that’s the story

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#28

Post by eridan2 »

FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

Fugaku supercomputer had a total cost of 1B$, and 160k cpus, so the cost was less than 6500 per chip, or 3600$ per 1.5 TFlops

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#29

Post by fluffything »

FeepingCreature wrote: post #4

An A64FX CPU has 48 cores. 1 CPU/node means 96 cores on 2 nodes for $40k and ~6TFLOPS (3 per). Meanwhile the 128 thread Threadripper 3990X costs $3.6k, and benchs put it at 1.5TFLOPS in Linpack, which is less but not 10x less, and I don't know what the benchmark basis for the A64FX value is, so I suspect it's closer, especially since it says theoretical peak performance. ... Am I missing something?

> ... Am I missing something?

Memory BW? Power consumption ? What's the BW and the power consumption of 2 A64FX CPUs, and what's the BW and power consumption of 4x Threadripper 3990X that also deliver 6 TFLOP/s? You might need to add the power for cooling to the comparison as well if you plan to stock a full cluster with these.

Also, if your main metric is TFLOPs, you might want to add a single A100 to the comparison.... it probably crushes the Threadripper and the A64FX from the POV of TFLOPs, memory BW, and power consumption, but maybe not price.

Re: HPC Systems Special Offer: Two A64FX Nodes in a 2U for $40k

#30

Post by sjreese »

This is a business workhouse; think web hosting, HPC scientific programming OR any Bitcoin mining related business. You have a system that pays for itself and RHEL means it will run any Linux application. Adding AI ( What is processing ) it is a real money maker in the USA. 2U means at home - I'll bet the US FTC is already placing import restrictions on it as we speak, (with AT&T wavier) See Also: SKYDRIVE https://nerdist.com/article/japanese-flying-cars-nerdist-new...
Post reply on HN