Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inside the Titan Supercomputer: 299K AMD x86 Cores and 18.6K Nvidia GPU Cores

anandtech.com

Post reply on HN
Print view View original

1–10 of 29 posts

Re: Inside the Titan Supercomputer: 299K AMD x86 Cores and 18.6K Nvidia GPU Cores

#2

Post by hendzen »

This is pretty awe-inspiring but as a programmer I know it would be fairly difficult to use this machine for existing workloads because so much code would have to be rewritten from typical x86 code to CUDA/OpenCL to use all those GPUs.

Personally, I'm more excited for the next wave of supercomputers built with racks of Xeon Phis [1].

[1] - http://www.intel.com/content/www/us/en/high-performance-comp...

Re: Inside the Titan Supercomputer: 299K AMD x86 Cores and 18.6K Nvidia GPU Cores

#4

Post by Osmium »

hendzen wrote: post #2

This is pretty awe-inspiring but as a programmer I know it would be fairly difficult to use this machine for existing workloads because so much code would have to be rewritten from typical x86 code to CUDA/OpenCL to use all those GPUs. Personally, I'm more excited for the next wave of supercomputers built with racks of Xeon Phis [1]. [1] - http://www.intel.com/content/www/us/en/high-performance-comp...

In fairness, I don't think there are any existing workloads that would benefit 299k cores that aren't already massively parallel :) If you need this kinda thing, your code is already going to be ready for it.

I'm with you on the Phis though. Can't wait for one to be affordable for a home machine

Re: Inside the Titan Supercomputer: 299K AMD x86 Cores and 18.6K Nvidia GPU Cores

#6

Post by tmurray »

hendzen wrote: post #2

This is pretty awe-inspiring but as a programmer I know it would be fairly difficult to use this machine for existing workloads because so much code would have to be rewritten from typical x86 code to CUDA/OpenCL to use all those GPUs. Personally, I'm more excited for the next wave of supercomputers built with racks of Xeon Phis [1]. [1] - http://www.intel.com/content/www/us/en/high-performance-comp...

(full disclosure: used to work for NV on CUDA and did very extensive work on Titan, so I am probably biased)

If you think your existing MPI app is going to automatically scale to a heterogeneous architecture (high-power x86 on the main CPU, Xeon Phi cores on the accelerator) and get acceptable performance, sorry, it's not going to happen.

The fundamental constraints on 2012/2013 Xeon Phi performance that determine how apps should be written are exactly the same as current desktop GPUs (small, high-latency local memory that is not coherent with the rest of the system; relatively slow, high-latency link to CPU; ugly interactions with network cards in most environments; fundamental need to hide memory latency at all times). For any sort of performance beyond a standard Xeon, you're going to want to run a Xeon Phi as a targeted accelerator rather than offloading entire processes to it and using a standard MPI stack. This means you're going to be running in a hybrid host/device mode and using compiler directives or a specific parallel language and API to deal with on-chip execution and data transfer, which puts you in exactly the same solution space as with GPUs.

in other words: the Phi of today is not a panacea. you get better tools and more flexibility in terms of the programming model, but the fast path that any of its intended market would use in applications looks identical to GPUs.

Re: Inside the Titan Supercomputer: 299K AMD x86 Cores and 18.6K Nvidia GPU Cores

#7

Post by paulsutter »

Does anyone know why they have a separate disk IO system when they could more easily just plug drives into each node/motherboard for higher aggregate throughout, less complexity, and a lower overall cost?

EDIT: Blade systems or no, the drives have to physically be placed somewhere. Having a separate subsystem can only take up more space, not less. Two reasons I can think of: (1) independent scaling of compute and storage, and (2) lack of software for a distributed filesystem. Most likely (2) plus inertia is the real reason, all the others seem like rationalizations. For example, they are either able to take nodes offline or they aren't. The need exists whether or not the disks are attached there.

Re: Inside the Titan Supercomputer: 299K AMD x86 Cores and 18.6K Nvidia GPU Cores

#8

Post by tmurray »

paulsutter wrote: post #7

Does anyone know why they have a separate disk IO system when they could more easily just plug drives into each node/motherboard for higher aggregate throughout, less complexity, and a lower overall cost? EDIT: Blade systems or no, the drives have to physically be placed somewhere. Having a separate subsystem can only take up more space, not less. Two reasons I can think of: (1) independent scaling of compute and sto…

They're blade systems, there's not really room for disks. Easier to keep it centralized and easily serviced.

Re: Inside the Titan Supercomputer: 299K AMD x86 Cores and 18.6K Nvidia GPU Cores

#9

Post by T-Winsnes »

paulsutter wrote: post #7

Does anyone know why they have a separate disk IO system when they could more easily just plug drives into each node/motherboard for higher aggregate throughout, less complexity, and a lower overall cost? EDIT: Blade systems or no, the drives have to physically be placed somewhere. Having a separate subsystem can only take up more space, not less. Two reasons I can think of: (1) independent scaling of compute and sto…

My guess is that most of the work they do on this machine won't be bottlenecked on the hdds, so they don't worry about it too much. And it's easier to replace hard drives in a separate rack than taking blades offline to do it.
Post reply on HN