Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Combine Multiple AWS Instances into a 16-GPU Monster Machine

bitfusion.io

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#21

Post by mtweak »

semi-extrinsic wrote: post #20
mtweak wrote: post #8

Earlier quoted context omitted.

Yes! Whenever you spin up one of our AMIs, there is a README that will guide you through a couple of simple examples. We are about to publish performance results on the monster machines in a few days, so watch out for it. Scaling depends on the compute density of the GPU workload, but in general we've seen pretty good results with 1) Deep learning (caffe) scaling to 16 GPUs (near native scaling with local GPUs, espec…

Have you done any molecular dynamics benchmarks? If so, what kind/what system? I'd be very interested to see those. If you haven't, I could probably contribute some strong and weak scaling testcases.

We've only done cursory evaluation of NAMD scaling. We saw a 7X improvement going from a non-GPU system to remote GPUs located in a different datacenter over shared 10g. We're not sure if that was with a representative dataset (MD is not our skill set), so if you can help us with a case study we'd be excited to work with you. Please do contact me.

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#22

Post by semi-extrinsic »

mtweak wrote: post #21
semi-extrinsic wrote: post #20

Earlier quoted context omitted.

Have you done any molecular dynamics benchmarks? If so, what kind/what system? I'd be very interested to see those. If you haven't, I could probably contribute some strong and weak scaling testcases.

We've only done cursory evaluation of NAMD scaling. We saw a 7X improvement going from a non-GPU system to remote GPUs located in a different datacenter over shared 10g. We're not sure if that was with a representative dataset (MD is not our skill set), so if you can help us with a case study we'd be excited to work with you. Please do contact me.

I sent you a message with some more info and my email via the contact form at bitfusion.io

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#23

Post by LoSboccacc »

derefr wrote: post #10
mtweak wrote: post #7

Earlier quoted context omitted.

Yes, your app would have to support multiple GPUs. What's done here is remoting CUDA/OpenCL/etc. calls so that remote GPUs can be accessed from a single instance. When performing device/platform enumeration, all GPUs appear to be directly connected to a single instance -- hence no change to the application required.

Sounds like Plan9's concept of "CPU server mounts" has been reborn as "GPU server mounts." Could actually get traction this time, given that existing multi-GPU programs will Just Work.

I cant wait for company to provide opencl/cuda mflops as a service instead of giving you vms as a whole, so one could just attach remote engine to any smallish controller vm

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#24

Post by LoSboccacc »

mchahn wrote: post #4

At first I thought this was the same problem as automatically breaking up apps to run in multiple cpus. This problem has been heavily researched with no success. Is it the fact that GPU code already runs in parallel streams that makes this possible?

Also that gpu have little bandwith toward main memory and most cuda patterns involve loading, computing and retrieving which works as well remotely as locally, as opposed to programs accessing memory at random time all over their memory space the gpu memory is neatly organized in texture areas and the programming paradigm already entails moving them in and out the device as few times as possible

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#25

Post by ant6n »

manav wrote: post #14

I've found amazon GPU instances to be really expensive (even the spot prices have been high recently), especially if you need it for longer runs for deep learning. The other issue is that the additional layers of virtualization create bandwidth overhead issues. I'd like to see something in the cloud thats bare-metal / full access to GPUs (Maybe a good idea to start one). For scaling higher with a very large number of…

Somebody should make a startup that allows people to sell access to their computers by the minute. Like spot instances in the cloud ... in people's basements. The true sharing economy.

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#26

Post by manav »

ant6n wrote: post #25
manav wrote: post #14

I've found amazon GPU instances to be really expensive (even the spot prices have been high recently), especially if you need it for longer runs for deep learning. The other issue is that the additional layers of virtualization create bandwidth overhead issues. I'd like to see something in the cloud thats bare-metal / full access to GPUs (Maybe a good idea to start one). For scaling higher with a very large number of…

Somebody should make a startup that allows people to sell access to their computers by the minute. Like spot instances in the cloud ... in people's basements. The true sharing economy.

IO is the bottleneck there for most applications, but there are some massively distributed grid computing projects that run on BOINC like Seti@HOME.

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#27

Post by eximius »

ant6n wrote: post #25
manav wrote: post #14

I've found amazon GPU instances to be really expensive (even the spot prices have been high recently), especially if you need it for longer runs for deep learning. The other issue is that the additional layers of virtualization create bandwidth overhead issues. I'd like to see something in the cloud thats bare-metal / full access to GPUs (Maybe a good idea to start one). For scaling higher with a very large number of…

Somebody should make a startup that allows people to sell access to their computers by the minute. Like spot instances in the cloud ... in people's basements. The true sharing economy.

That sounds like a nightmare. You'd have zero uptime guarantees!

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#28

Post by rm999 »

manav wrote: post #14

I've found amazon GPU instances to be really expensive (even the spot prices have been high recently), especially if you need it for longer runs for deep learning. The other issue is that the additional layers of virtualization create bandwidth overhead issues. I'd like to see something in the cloud thats bare-metal / full access to GPUs (Maybe a good idea to start one). For scaling higher with a very large number of…

I did the analysis about 9 months ago for my team when the 980 ti's came out, and the AWS pricing was expensive (we built a server that paid for itself in 2 weeks compared to a g2.8xlarge). This is largely because the 980 ti is actually a ridiculously good deal for price/performance.

The bigger problem we ran into is that AWS instances use a lot of small GPUs, which don't scale well using a lot of deep neural network tools (e.g. theano). It was never really a viable option for us.

Re: Combine Multiple AWS Instances into a 16-GPU Monster Machine

#30

Post by Razengan »

ant6n wrote: post #25
manav wrote: post #14

I've found amazon GPU instances to be really expensive (even the spot prices have been high recently), especially if you need it for longer runs for deep learning. The other issue is that the additional layers of virtualization create bandwidth overhead issues. I'd like to see something in the cloud thats bare-metal / full access to GPUs (Maybe a good idea to start one). For scaling higher with a very large number of…

Somebody should make a startup that allows people to sell access to their computers by the minute. Like spot instances in the cloud ... in people's basements. The true sharing economy.

Why not just move forward to realizing some sort of distributed/decentralized internet?

Something like a combination of Freenet, TOR, BOINC, blockchain etc. technologies, using the current "legacy" internet as a backbone, where anyone can voluntarily offer their computing and storage resources to the network at varying levels of participation.

Say you could offer your laptop as a simple discovery/directory node to simply help others connect and find stuff, and your desktop as either a static-content serving node or as a computation node that can host distributed applications, like SETI@home or web apps like Facebook.

Maybe even reward cryptocurrency to those who offer the most resources.

Post reply on HN