Microsoft Readies Azure GPUs • Hacker News Board

Re: Microsoft Readies Azure GPUs

#11

Post by jjallen » Thu, Aug 11, 2016, 2:58 PM UTC

dharma1 wrote: post #8

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

it sure does. I hope AWS and GCE will step up their GPU instance game

GCE should offer Tensorflow instances sometime soon. Closed alpha now. This will be a nice offering from Google which is no doubt part of the reason they open sourced Tensorflow.

Which GPUs they offer and how much, nobody knows.

Re: Microsoft Readies Azure GPUs

#12

Post by ryan-c » Thu, Aug 11, 2016, 2:59 PM UTC

Eridrus wrote: post #10

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

Does Azure offer spot pricing? You can get those AWS instances for (iirc) 10-15 cents/hr without much interruption.

The spot pricing seems to go up pretty quickly if you try to run more than one instance.

Re: Microsoft Readies Azure GPUs

#13

Post by blahi » Thu, Aug 11, 2016, 2:59 PM UTC

JoshGlazebrook wrote: post #5

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

I wish one of the main cloud platforms would significantly cut their per/GB bandwidth costs. It feels like there has been no adjustments in years. Also funny story, I was 14 or 15 and was trying to figure out a name for a stupid little chat client I was trying to make in C#, and somehow azure popped into my head. Not surprisingly azure.com was taken and just had a generic microsoft landing page (this was a little bef…

That's a major lock-in mechanism. Data stays in for the compute. Which is a major pity for Microsoft since all the marketing measurement startups got built on top of AWS while Azure has superior analytics tools.

Re: Microsoft Readies Azure GPUs

#14

Post by dogma1138 » Thu, Aug 11, 2016, 3:03 PM UTC

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

Just a small correction the K80 is a dual GPU setup which means you get 2 GPU's with 2496 "CUDA cores" each and 12GB assigned to each GPU.

This isn't the same as having 1 GPU with 4992 cores and 24GB of memory.

CUDA kernel scaling is pretty decent but it's not 1:1 with additional GPU's it's also very much task specific so tasks that cannot be easily parallelized will not scale well or at all at least not without you splitting the workload manually. As for the memory scaling CUDA supports P2P memcopy so you can easily move memory from 1 GPU to another (or between hosts in a network distributed configuration) but under best case scenarios this goes through a PCIE bridge (in the K80 it's onboard) but no NVLINK support on the K80 as it's 2 gens behind Pascal, if you are using P2P memcopy you do not double the memory capacity (even lose some since memcopy requires another buffer iirc). CUDA also allows you to share the memory directly between multiple GPU's (and the CPU/Host) NVIDIA calls this Unified Addressing (OpenCL 2.0 later implemented something similar as Shared Virtual Memory) but this also comes at some cost the latency is greatly increased for memory access and you really need stellar code to make everything work properly if 1 GPU tries to dereference a pointer on another GPU while it's still is in use whelp "don't panic and move on" probably won't work :P

Re: Microsoft Readies Azure GPUs

#15

Post by Athas » Thu, Aug 11, 2016, 3:04 PM UTC

This is interesting. Is there some way for academics to get discounted access for experiments (not for running heavy simulations, but for testing whether our stuff works)?

Re: Microsoft Readies Azure GPUs

#16

Post by dogma1138 » Thu, Aug 11, 2016, 3:08 PM UTC

Athas wrote: post #15

This is interesting. Is there some way for academics to get discounted access for experiments (not for running heavy simulations, but for testing whether our stuff works)?

Microsoft has graned "free" subscriptions to proffessors and researchers before, so it can't hurt to ask :P https://azure.microsoft.com/en-gb/community/education/

Re: Microsoft Readies Azure GPUs

#17

Post by snuxoll » Thu, Aug 11, 2016, 3:08 PM UTC

JoshGlazebrook wrote: post #5

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

I wish one of the main cloud platforms would significantly cut their per/GB bandwidth costs. It feels like there has been no adjustments in years. Also funny story, I was 14 or 15 and was trying to figure out a name for a stupid little chat client I was trying to make in C#, and somehow azure popped into my head. Not surprisingly azure.com was taken and just had a generic microsoft landing page (this was a little bef…

I mean, it doesn't necessarily solve anything if you're using it for public facing apps, but if you're doing a hybrid solution with Azure you can buy unmetered ExpressRoute ports and at least avoid the metered bandwidth costs back to a datacenter.

Re: Microsoft Readies Azure GPUs

#18

Post by n00b101 » Thu, Aug 11, 2016, 3:08 PM UTC

dogma1138 wrote: post #14

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

Just a small correction the K80 is a dual GPU setup which means you get 2 GPU's with 2496 "CUDA cores" each and 12GB assigned to each GPU. This isn't the same as having 1 GPU with 4992 cores and 24GB of memory. CUDA kernel scaling is pretty decent but it's not 1:1 with additional GPU's it's also very much task specific so tasks that cannot be easily parallelized will not scale well or at all at least not without you…

Thanks. I was glossing over the details. The point I was trying to emphasize is that the Azure instance gives you the whole K80 board (i.e. two GK210 processors), unlike the AWS G2 instance which effectively gives you a virtualized half of a K10 board (i.e. one GK104 processor)

Re: Microsoft Readies Azure GPUs

#19

Post by gcr » Thu, Aug 11, 2016, 3:11 PM UTC

n00b101 wrote: post #4

Not sure about the exact pricing shown here, but it looks like Azure blows AWS GPU pricing out of the water. Comparing specs: AWS g2.2xlarge: * DRAM: 15GB * GPU processor: 1xGK104 * 1536 CUDA cores * 2.29Tflops/s peak single precision throughput * GPU memory: 4GB * 160GB/s peak bandwidth * Price: $0.65/hr (US East, Linux) Azure NC6: * DRAM: 56GB * GPU processors: 2xGK210 * 4992 CUDA cores * 8.73Tflops/s peak single p…

What about comparing against spot market prices? Those AWS GPU instances are frequently $0.10 or $0.20 rather than $0.66. Not clearly a better deal anymore.

Re: Microsoft Readies Azure GPUs

#20

Post by n00b101 » Thu, Aug 11, 2016, 3:12 PM UTC

jjallen wrote: post #11

dharma1 wrote: post #8

Earlier quoted context omitted.

it sure does. I hope AWS and GCE will step up their GPU instance game

GCE should offer Tensorflow instances sometime soon. Closed alpha now. This will be a nice offering from Google which is no doubt part of the reason they open sourced Tensorflow. Which GPUs they offer and how much, nobody knows.

> Which GPUs they offer and how much, nobody knows.

They might not offer GPUs at all. It may be that Google is planning to build a cloud using their own proprietary "Tensor Processing Unit" chip designed for TensorFlow (https://en.wikipedia.org/wiki/Tensor_processing_unit)