Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Outperforms GCP in the 2018 Cloud Report

cockroachlabs.com

Post reply on HN
Print view View original

21–30 of 104 posts

Re: AWS Outperforms GCP in the 2018 Cloud Report

#21

Post by halbritt »

WestCoastJustin wrote: post #15

There might be a networking cap & disk I/O issues with the instances you picked on GCP vs AWS. The GCP instance has 8 Gbps vs 10 Gbps for AWS. I don't really know without seeing the graphs from the instances, if you hit a cap, but this could make a difference in both transfer speeds and latency #'s for GCP. Also, for your local disk test, on GCP, disk size makes a difference to get the best performance. The larger th…

I don't understand the throughput numbers given. 5.6GB/s for GCP and 9.6GB/s for AWS would be 44gbps and 76gbps respectively.

I don't don't know of any instances offering that kind of throughput.

I've personally validated GCP's statement that they offer 2gbps/core up to 16gbps. I can get 16gbps consistently between any two n1-standard-8 using iperf.

This generally makes network IO in GCP much cheaper.

Re: AWS Outperforms GCP in the 2018 Cloud Report

#22

Post by peferron »

WestCoastJustin wrote: post #15

There might be a networking cap & disk I/O issues with the instances you picked on GCP vs AWS. The GCP instance has 8 Gbps vs 10 Gbps for AWS. I don't really know without seeing the graphs from the instances, if you hit a cap, but this could make a difference in both transfer speeds and latency #'s for GCP. Also, for your local disk test, on GCP, disk size makes a difference to get the best performance. The larger th…

> Also, for your local disk test, on GCP, disk size makes a difference to get the best performance. The larger the disk, the better the performance. Disk read/write performance also comes out of the available network bandwidth

Do you have a source for local SSD performance coming out of the available network bandwidth? According to GCP docs [1], this only applies to persistent disks. Local SSD perf depends only on disk size and choice of SCSI/NVMe interface.

According to another GCP doc [2], local SSDs are all 375 GB in size. For comparison, c5d.4xlarge has 400 GB, which is very close. So I don't see anything wrong in the benchmark unless they messed up and ran it against the persistent root disk instead of the local SSD.

[1] https://cloud.google.com/compute/docs/disks/performance#type...

[2] https://cloud.google.com/compute/docs/disks/#localssds

Re: AWS Outperforms GCP in the 2018 Cloud Report

#23

Post by WestCoastJustin »

peferron wrote: post #22
WestCoastJustin wrote: post #15

There might be a networking cap & disk I/O issues with the instances you picked on GCP vs AWS. The GCP instance has 8 Gbps vs 10 Gbps for AWS. I don't really know without seeing the graphs from the instances, if you hit a cap, but this could make a difference in both transfer speeds and latency #'s for GCP. Also, for your local disk test, on GCP, disk size makes a difference to get the best performance. The larger th…

> Also, for your local disk test, on GCP, disk size makes a difference to get the best performance. The larger the disk, the better the performance. Disk read/write performance also comes out of the available network bandwidth Do you have a source for local SSD performance coming out of the available network bandwidth? According to GCP docs [1], this only applies to persistent disks. Local SSD perf depends only on di…

You are right. Sorry for the confusion. It is only PD (Persistent Disk) that comes out of network bandwidth. Anything on the NVMe SSD would be totally local to the machine (no network caps, etc). The article doesn't really say if they are using SSD PD or SSD NVMe for GCP. Also, disk size does matter for the NVMe SSD and performance (as you can stripe them together by adding more; up to 4). You can see the #'s by using the console and playing around with adding more NVMe SSDs (via this doc [1]).

  Size      Random IOPS                Throughput limit (MB/s)
  375GB     169,987 (r)  90,000 (w)      663 (r)   352 (w)
  750GB     339,975 (r) 180,000 (w)    1,327 (r)   705 (w)
  1125GB    509,962 (r) 270,000 (w)    1,991 (r) 1,057 (w)
  1500GB    679,950 (r) 360,000 (w)    2,650 (r) 1,400 (w)
[1] https://cloud.google.com/compute/docs/disks/local-ssd

Re: AWS Outperforms GCP in the 2018 Cloud Report

#24

Post by jsnell »

halbritt wrote: post #21
WestCoastJustin wrote: post #15

There might be a networking cap & disk I/O issues with the instances you picked on GCP vs AWS. The GCP instance has 8 Gbps vs 10 Gbps for AWS. I don't really know without seeing the graphs from the instances, if you hit a cap, but this could make a difference in both transfer speeds and latency #'s for GCP. Also, for your local disk test, on GCP, disk size makes a difference to get the best performance. The larger th…

I don't understand the throughput numbers given. 5.6GB/s for GCP and 9.6GB/s for AWS would be 44gbps and 76gbps respectively. I don't don't know of any instances offering that kind of throughput. I've personally validated GCP's statement that they offer 2gbps/core up to 16gbps. I can get 16gbps consistently between any two n1-standard-8 using iperf. This generally makes network IO in GCP much cheaper.

I wouldn't read too much into it, it's clearly Gbps. The authors are just sloppy with capitalization. They also elsewhere talk about iperf having "128 kb" buffer which seems unlikely, and the throughput graph says "gb" where the text says "GB".

And then there's "iPerf" and "PING"...

Re: AWS Outperforms GCP in the 2018 Cloud Report

#25

Post by adrr »

brian_cunnie wrote: post #13

I've also benchmarked GCP vs. AWS [0], and, for the tests that I ran, found that GCP outperformed AWS by a factor of 3:1. Specifically, a GCP instance n1-highcpu-8 with a 256GB pd-ssd disk, clocked in at 11,728 IOPS vs an AWS c4.xlarge with a 256GB gp2 disk, clocking in at 3,634 IOPS. To put that in context of the blog post, it means your setup can drastically affect your results. Using local NVMe disk, for example,…

If you need raw speed for storage why mess around with EBS or any other network storage device. Your case with videos. You could shove the videos for permanent storage on S3 and then use I2 instance and it's ephemeral storage to serve videos. I think I2 NVME drives push 250,000 IOPS per drive and you can get up to 8 drives per instance.

Re: AWS Outperforms GCP in the 2018 Cloud Report

#26

Post by kyrra »

awoods187 wrote: post #12
kyrra wrote: post #9

Is this table labeled wrong? https://d33wubrfki0l68.cloudfront.net/f61cd6683f5c13f8d2b506... All the text around it says GCP is worse, but the table shows GCP is better.

The labels are incorrect (we combined the charts in the final draft)--fix incoming. GCP was much worse than AWS on this test.

Looks fixed now. Thanks.

Re: AWS Outperforms GCP in the 2018 Cloud Report

#27

Post by peferron »

virtuallynathan wrote: post #17
brian_cunnie wrote: post #13

I've also benchmarked GCP vs. AWS [0], and, for the tests that I ran, found that GCP outperformed AWS by a factor of 3:1. Specifically, a GCP instance n1-highcpu-8 with a 256GB pd-ssd disk, clocked in at 11,728 IOPS vs an AWS c4.xlarge with a 256GB gp2 disk, clocking in at 3,634 IOPS. To put that in context of the blog post, it means your setup can drastically affect your results. Using local NVMe disk, for example,…

I would compare against the C5 instance type; it uses the newer "Nitro" hypervisor.

Just looking at the report's second graph, c5d.4xlarge has twice the throughput of i3.4xlarge, which is incredible. I wish AWS would release a new generation of "i" instances, with the same large amount of local storage as i3, but built on the same platform (including Nitro and processor choices) as c5/m5.

Re: AWS Outperforms GCP in the 2018 Cloud Report

#28

Post by briffle »

nodesocket wrote: post #19

Developer experience on GCP is vastly superior to AWS. - Pricing on GCP is much easier, no need to purchase reserved instances, figure out all the details and buried AWS billing rules. Run your GCP instances and automatically get discounts. AWS reserved instances requires knowing your instance types, knowing that you can purchase the smallest type of an instance class and combine, knowing that you can only purchase 2…

Another great point to GCP: -- You can setup an 'organization' that will hold multiple projects, in a hierarchy of folders. You can set permissions/roles on a folder, and have them propagate down to all projects underneath.

Re: AWS Outperforms GCP in the 2018 Cloud Report

#29

Post by sethvargo »

Hey all - Seth from Google here.

Thank you to the authors who worked on this report. These types of reports help us better understand the ways in which our customers and partners utilize our platform. Our team is reviewing the report and will provide a response as we conduct our own benchmarks.

Varying factors impact these types of benchmark analyses, many of which are difficult to isolate and control. As an example, I refer to some of the benchmarks others have posted in this very thread.

As a technical practitioner, I'm positive there are areas in which cloud A outperforms cloud B and vice versa, giving users choice and flexibility. As an employee of Google, I can assure you that we are committed to providing best in class performance and availability on our platform.

Thank you for your patience as we review these findings and craft our responses.

Re: AWS Outperforms GCP in the 2018 Cloud Report

#30

Post by KirinDave »

kharms wrote: post #18

For someone who has worked with both: which offers a better developer onboarding experience, for small web/data apps? Ease of learning and use wise.

There are many dimensions in which working with GCP is a lot better for small and medium apps, if only the proximity to Firebase.

These results are, to my mind, quite questionable. They certainly don't line up with my personal experience and the measurements I've done in the last year.

AWS is much harder to use correctly right now, to me.

Post reply on HN