Microsoft Readies Azure GPUs • Hacker News Board

Re: Microsoft Readies Azure GPUs

#21

Post by dogma1138 » Thu, Aug 11, 2016, 3:13 PM UTC

wjoe wrote: post #9

This is pretty interesting from a video encoding point of view. We're currently using g2.2xlarge AWS instances which are Kepler based. All we really need from those instances is the GPU (more specifically, the NVENC H264 encoding chip), so other specs aren't too important. It looks like the NV6 Azure instances use a newer Maxwell based chip, which can handle much more (concurrent, or higher bitrate/resolution) videos…

Just for reference:

K1 (AWS): 1 GPUs x 6 H.264 streams (max single stream 720p@30fps)

K2 (Azure K80): 1 GPUs x 6 H.264 streams max single stream 720p@30fps)

M6 (Don't know if it's used by anyone ;)): 1 GPU x 18 H.264 streams (max single stream 1080p@30fps)

M60 (Azure M60): 1 GPU x 18 H.264 streams (max single stream 1080p@30fps)

You can sacrifice encoder bandwidth (max streams) for higher resolutions or frame rates.

Re: Microsoft Readies Azure GPUs

#22

Post by gcr » Thu, Aug 11, 2016, 3:14 PM UTC

ryan-c wrote: post #12

Eridrus wrote: post #10

Earlier quoted context omitted.

Does Azure offer spot pricing? You can get those AWS instances for (iirc) 10-15 cents/hr without much interruption.

The spot pricing seems to go up pretty quickly if you try to run more than one instance.

I've run 6 of those GPU instances for $0.60ish/hr.

Pricing does go up near computer vision conference deadlines though. Many research labs are too poor to buy our own hardware. :-)

Re: Microsoft Readies Azure GPUs

#23

Post by dogma1138 » Thu, Aug 11, 2016, 3:21 PM UTC

n00b101 wrote: post #18

dogma1138 wrote: post #14

Earlier quoted context omitted.

Just a small correction the K80 is a dual GPU setup which means you get 2 GPU's with 2496 "CUDA cores" each and 12GB assigned to each GPU. This isn't the same as having 1 GPU with 4992 cores and 24GB of memory. CUDA kernel scaling is pretty decent but it's not 1:1 with additional GPU's it's also very much task specific so tasks that cannot be easily parallelized will not scale well or at all at least not without you…

Thanks. I was glossing over the details. The point I was trying to emphasize is that the Azure instance gives you the whole K80 board (i.e. two GK210 processors), unlike the AWS G2 instance which effectively gives you a virtualized half of a K10 board (i.e. one GK104 processor)

Oh yeah the Kepler 2 K80's blow the K10 out of the water. I wonder just how expensive the enterprise grade Pascal boards are that even Microsoft doesn't offer them as an option.

It's really amazing just how much money NVIDIA seems to be squeezing out of the compute market (almost makes you feel bad for AMD) especially considering that Kepler is a 4 almost 5 year old architecture by now. The K80 is still going for like 4000-5000$ a pop I would love to know just how many GPU's Microsoft ended up buying.

Re: Microsoft Readies Azure GPUs

#24

Post by pilooch » Thu, Aug 11, 2016, 3:24 PM UTC

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

Well, check this one out: https://www.runabove.com/titan-x-gpu-servers.xml

Re: Microsoft Readies Azure GPUs

#25

Post by LogicFailsMe » Thu, Aug 11, 2016, 3:26 PM UTC

AWS: 4 year-old GPUs

Azure: 2 year-old GPUs

Wouldn't it be awesome if a cloud provider could keep up with the GPU roadmap and ship something within a year or so of its release?

Re: Microsoft Readies Azure GPUs

#26

Post by KirinDave » Thu, Aug 11, 2016, 3:26 PM UTC

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

Have they historically done so though? AWS's market leader position seems to persist even though they're actually quite a bit more expensive than Google Cloud and Azure (in particular due to the billing practices).

Re: Microsoft Readies Azure GPUs

#27

Post by KirinDave » Thu, Aug 11, 2016, 3:29 PM UTC

Eridrus wrote: post #10

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

Does Azure offer spot pricing? You can get those AWS instances for (iirc) 10-15 cents/hr without much interruption.

No, but it doesn't do hourly billing so it can still be cheaper depending on your workload.

Also, you tend to be getting more per instance hour on both Azure and GCE, assuming you do things those environments are good at. In Azure's case if you can get your software running on windows then it's actually quite good performance for I/O intensive things.

Re: Microsoft Readies Azure GPUs

#28

Post by dharma1 » Thu, Aug 11, 2016, 3:34 PM UTC

dogma1138 wrote: post #23

n00b101 wrote: post #18

Earlier quoted context omitted.

Thanks. I was glossing over the details. The point I was trying to emphasize is that the Azure instance gives you the whole K80 board (i.e. two GK210 processors), unlike the AWS G2 instance which effectively gives you a virtualized half of a K10 board (i.e. one GK104 processor)

Oh yeah the Kepler 2 K80's blow the K10 out of the water. I wonder just how expensive the enterprise grade Pascal boards are that even Microsoft doesn't offer them as an option. It's really amazing just how much money NVIDIA seems to be squeezing out of the compute market (almost makes you feel bad for AMD) especially considering that Kepler is a 4 almost 5 year old architecture by now. The K80 is still going for lik…

Is there any reason cloud providers don't offer virtualised Titan X, or GTX 1080?

Re: Microsoft Readies Azure GPUs

#29

Post by dharma1 » Thu, Aug 11, 2016, 3:35 PM UTC

pilooch wrote: post #24

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

Well, check this one out: https://www.runabove.com/titan-x-gpu-servers.xml

That's nice, but the whole point of cloud instances is to be able to spin them up and down in large numbers when you need them, and pay by the hour

Re: Microsoft Readies Azure GPUs

#30

Post by dharma1 » Thu, Aug 11, 2016, 3:37 PM UTC

KirinDave wrote: post #27

Eridrus wrote: post #10

Earlier quoted context omitted.

Does Azure offer spot pricing? You can get those AWS instances for (iirc) 10-15 cents/hr without much interruption.

No, but it doesn't do hourly billing so it can still be cheaper depending on your workload. Also, you tend to be getting more per instance hour on both Azure and GCE, assuming you do things those environments are good at. In Azure's case if you can get your software running on windows then it's actually quite good performance for I/O intensive things.

Why Windows? You can run Linux on Azure