Live data from Hacker News

Why Cloudflare Chose AMD EPYC for Gen X Servers

blog.cloudflare.com

111–120 of 123 posts

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#111
post #92

Earlier quoted context omitted.

I'm curious why requests per watt is an important metric? Are you running up against some sort of heat limit or optimizing for electricity cost? Seems like requests per amortized dollar all in would make more sense. Or requests per cubic foot of data center space if that is your limitation.

This is a good question and I would like to learn the answer too. According to some napkin calculations, the CPU is $5000 and will be kept in service, I guess, 5 years. The US average price is 13 cents / kWh so this 225 watts CPU will cost $1280 in electricity over those 5 years. Colocation price is probably low, let's say $50 / 1U server / month. So that's another $3000. Why is electricity the most important here? I…

Data center space is generally plentiful. Power and cooling, which for most purposes can be considered equivalent, are the scarce resource.

CloudFlare is trading half their server density[0] for a ~25% gain in performance-per-watt and performance-per-node[1]. Power is the whole ball-game at scale.

[0] https://blog.cloudflare.com/a-tour-inside-cloudflares-g9-ser...

[1] https://blog.cloudflare.com/an-epyc-trip-to-rome-amd-is-clou...

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#112

Earlier quoted context omitted.

Tools can't close a gap of entire cores, much less twice or more of them.

It's a matter of principle. Some people don't want cores that you can't gather insight into how you are spending/wasting their time. I'm not a direct customer of Intel or AMD, but in general I would trade off clock speed/cores/cost for insight into what my software is doing.

[deleted]

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#113

Earlier quoted context omitted.

The rr debugger, which is an amazing productivity enhancer for C/C++ development, still doesn't support AMD: https://github.com/mozilla/rr/issues/2034

i wonder why doesn't AMD sponsor these kind of projects to get maintainers to support their CPU. it helps both parties.

If you look at the rr issue, there is plenty of interest from the rr maintainers (of whom I am one), and also some engagement from AMD. The problem is there is a deep bug in AMD's performance counters or their kernel PMU driver, it's really hard to figure out, and the AMD staff best placed to figure it out are (I assume) really busy.

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#114
post #111

Earlier quoted context omitted.

This is a good question and I would like to learn the answer too. According to some napkin calculations, the CPU is $5000 and will be kept in service, I guess, 5 years. The US average price is 13 cents / kWh so this 225 watts CPU will cost $1280 in electricity over those 5 years. Colocation price is probably low, let's say $50 / 1U server / month. So that's another $3000. Why is electricity the most important here? I…

Data center space is generally plentiful. Power and cooling, which for most purposes can be considered equivalent, are the scarce resource. CloudFlare is trading half their server density[0] for a ~25% gain in performance-per-watt and performance-per-node[1]. Power is the whole ball-game at scale. [0] https://blog.cloudflare.com/a-tour-inside-cloudflares-g9-ser... [1] https://blog.cloudflare.com/an-epyc-trip-to-rome-…

Ah, so it's not about power being a big expense, but being a constraint that is limiting their growth:

> We are constrained by the data centers’ power infrastructure which, along with our selected power distribution units, leads us to power cap for each server rack.

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#115

Earlier quoted context omitted.

I have never seen anyone have any love for Itanium. It's such a beautiful architecture, but nobody seems to touch it and get out unharmed.

What about it is beautiful? There idea of VLIW?

Yes, the idea of optimising at compile time so you don't have to do OoO and get a simple but functional chip.

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#116
post #103

Earlier quoted context omitted.

The person above also pointed out that things like machine learning are often better done on a GPU anyways. Certainly if you offload your video encoding to your GPU it does change the value proposition of that entirely.

I'm not sure what you are trying to say exactly but it seems like you are making some sort of prediction of the future. Video encoding, especially newer codecs like HEVC and VP9 are CPU intensive and SIMD optimized tasks that people are doing right now.

I'm not referring to any sort of future prediction, those people can switch to GPU encoding of video right now and enjoy better performance.

https://developer.nvidia.com/video-encode-decode-gpu-support...

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#117
post #116

Earlier quoted context omitted.

I'm not sure what you are trying to say exactly but it seems like you are making some sort of prediction of the future. Video encoding, especially newer codecs like HEVC and VP9 are CPU intensive and SIMD optimized tasks that people are doing right now.

I'm not referring to any sort of future prediction, those people can switch to GPU encoding of video right now and enjoy better performance. https://developer.nvidia.com/video-encode-decode-gpu-support...

They asked what people are using it for right now and video encoding is one of those things. This is very direct. They didn't ask about solutions or what they should do about a particular problem, just about what people are currently doing.

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#118

Earlier quoted context omitted.

https://cpu.userbenchmark.com/Compare/Intel-Core-i7-9700K-vs... Comparing the similar priced Intel and AMD CPU's shows Intel has a slight advantage. Reading the conclusion, it seems for Overclocked performance, AMD is the winner, especially when factoring in large core workloads. As games and typical "home-user" scenarios are not designed to take advantage of large multi-core CPU's, Intel would appear to be the bette…

dont use this site. they do all kind of shady things to make Intel CPUs look better https://www.reddit.com/r/buildapc/comments/chco8h/userbenchm...

Thank you. I was not aware of that.

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#119

Earlier quoted context omitted.

What about it is beautiful? There idea of VLIW?

Yes, the idea of optimising at compile time so you don't have to do OoO and get a simple but functional chip.

I think my stance on that is https://news.ycombinator.com/item?id=22459095

Re: Why Cloudflare Chose AMD EPYC for Gen X Servers

#120
post #73

Earlier quoted context omitted.

I doubt tooling is the problem that is slowing down AMD adoption. Compiler or special applications writers care about that. These are not the ones who pay AMD for their platform. It's the big businesses - most care about stability, predictability, reliability hence they stay with Intel. Those that are interested mostly in bang/buck, like Cloudflare, choose to go with AMD, tooling or no tooling. You don't need tooling…

This just reminded me of some older article on Numpy performance on Intel vs AMD [1]. I’m not posting this to bash AMD. Just trying to make point that the software side can matter a lot and from end customer point-of-view this can make a huge difference. [1] https://www.pugetsystems.com/labs/hpc/AMD-Ryzen-3900X-vs-Int...

I was just going to mention numpy, or.. libc.

Sure, if AMD hardware is much better than Intels, an unoptimized libc might run better there. That’s not the point.

The point is that a libc optimized for AMD could run much better than what it does today.

Improving the performance of low level libraries for Intel hardware is trivial because there are a lot of great tools that help you with that.

If I had to improve the performance of some library on AMD hardware, I would get some Intel hardware and tools, improve the performance for Intel, and then “hope” that this improves the performance for AMD as well.

Post reply on HN