Earlier quoted context omitted.
I'm curious why requests per watt is an important metric? Are you running up against some sort of heat limit or optimizing for electricity cost? Seems like requests per amortized dollar all in would make more sense. Or requests per cubic foot of data center space if that is your limitation.
This is a good question and I would like to learn the answer too. According to some napkin calculations, the CPU is $5000 and will be kept in service, I guess, 5 years. The US average price is 13 cents / kWh so this 225 watts CPU will cost $1280 in electricity over those 5 years. Colocation price is probably low, let's say $50 / 1U server / month. So that's another $3000. Why is electricity the most important here? I…
CloudFlare is trading half their server density[0] for a ~25% gain in performance-per-watt and performance-per-node[1]. Power is the whole ball-game at scale.
[0] https://blog.cloudflare.com/a-tour-inside-cloudflares-g9-ser...
[1] https://blog.cloudflare.com/an-epyc-trip-to-rome-amd-is-clou...