Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An EPYC NixOS Build Farm

grahamc.com

Post reply on HN
Print view View original

21–30 of 37 posts

Re: An EPYC NixOS Build Farm

#21

Post by ksec »

tmd83 wrote: post #2

This is quite surprising. I assumed they would get an EPYC server with higher core at similar price (due to the cost of the CPU) and get advantage in performance/dollar. I didn't expect EPYC to beat a similar physical core count intel server running at similar core count let alone by this much margin. Am I missing something here? I'm saying this as someone who really wants AMD to succeed and push the ecosystem forwar…

>They are effectively getting 35% extra performance 60% of the price that's just beyond impressive.

Lets just ignore the price point for now because the memory capacity are different.

But the performance, 24 Core AMD EPYC Zen is ~35% faster than 24 Core Intel Broadwell Xeon under the same frequency? Broadwell isn't even that old because Intel has only launched Skylake Xeon E5 not long ago. And the different between Skylake and Broadwell IPC aren't that much either. the max turbo speed are similar, and the Intel platform had the advantage of two CPU using two heatsink, higher TDP, higher heat dissipation more time for turbo.

And most testing has shown Zen to be about Sandybridge IPC performance. Something seems wrong here ( Or may be too good to be true )

Re: An EPYC NixOS Build Farm

#22

Post by blattimwind »

kingbirdy wrote: post #20
blattimwind wrote: post #18

Earlier quoted context omitted.

> EPYC is compared to a previous generation Xeon with lower TDP ... Edit: actually, this is a dual Xeon machine so its TDP is slightly higher than EPYC's. Comparing TDPs between manufacturers is invalid.

> Comparing TDPs between manufacturers is invalid. Why? It's all Watts

"TDP" is not a figure resulting from a standardized process. Each manufacturer has a different (and sometimes changing) definition of it. E.g. for Intel it is a supposed power consumption given a certain load at base clock. Meanwhile with AMD it basically encodes the standard thermal parameters (Tamb, Tj(max), Rthjc).

Re: An EPYC NixOS Build Farm

#24

Post by grhmc »

chris_st wrote: post #9

I'm curious if anyone knows the test tool they're using here, in particular for lines like: $machine->waitForWindow("^Desktop "); It looks incredibly useful. Thanks!

The tooling used by NixOS is custom to NixOS. You can check out the test directory for many more examples.

I'll be writing more soon about how these tests work and how neat they are. They really help NixOS quickly release really high quality updates!

Re: An EPYC NixOS Build Farm

#25

Post by noneucat »

chris_st wrote: post #9

I'm curious if anyone knows the test tool they're using here, in particular for lines like: $machine->waitForWindow("^Desktop "); It looks incredibly useful. Thanks!

The tests are written in Perl and are invoked by Nix's build tools. You can skim the API at https://nixos.org/nixos/manual/#sec-nixos-tests

In this case, the test makes use of both xdotool and Nix's own testing library.

Re: An EPYC NixOS Build Farm

#26

Post by ris »

O_H_E wrote: post #10
chris_st wrote: post #9

I'm curious if anyone knows the test tool they're using here, in particular for lines like: $machine->waitForWindow("^Desktop "); It looks incredibly useful. Thanks!

Well there seems to be something called nix-shell, but I never heard of it before. It seams pretty useful. The link to plasma tests is provided in the article. https://github.com/NixOS/nixpkgs/blob/49a6964a4250d98644da61...

This isn't nix-shell. nix-shell is a tool for spawning ephemeral software environments composed of nix packages, something a little like python's virtualenvs.

Digging a little bit, nixos' tests appear to be written in perl and are executed in a context that has access to a bunch of useful primitives for asserting these sorts of things. https://nixos.org/nixos/manual/#sec-writing-nixos-tests

Re: An EPYC NixOS Build Farm

#27

Post by aseipp »

chris_st wrote: post #9

I'm curious if anyone knows the test tool they're using here, in particular for lines like: $machine->waitForWindow("^Desktop "); It looks incredibly useful. Thanks!

It's our custom tool. NixOS has testing infrastructure built on top of QEMU, and you can use Perl scripts like this in order to automate aspects of testing inside that environment by talking to the QEMU monitor -- for example, you can not only wait for a window, but click a button in that window, then also wait for a service to start afterwords, for example. (If you were testing GUI functionality).

Tests are written in a mix of Nix and Perl automation scripts like this; the Perl scripts control QEMU, but the Nix code describes what is put into the disk images. It's fairly bespoke and tuned to our own needs. The source code for the Perl driver is here: https://github.com/NixOS/nixpkgs/blob/f148c5c4a1ffa353836bfd...

It can even do multi-machine test setups as another comment pointed out, and it's pretty fun for writing functional demonstrations; for example I've been meaning to use these tools to write examples of doing PostgreSQL replication and failover, etc.

Re: An EPYC NixOS Build Farm

#28

Post by uluyol »

ksec wrote: post #21
tmd83 wrote: post #2

This is quite surprising. I assumed they would get an EPYC server with higher core at similar price (due to the cost of the CPU) and get advantage in performance/dollar. I didn't expect EPYC to beat a similar physical core count intel server running at similar core count let alone by this much margin. Am I missing something here? I'm saying this as someone who really wants AMD to succeed and push the ecosystem forwar…

>They are effectively getting 35% extra performance 60% of the price that's just beyond impressive. Lets just ignore the price point for now because the memory capacity are different. But the performance, 24 Core AMD EPYC Zen is ~35% faster than 24 Core Intel Broadwell Xeon under the same frequency? Broadwell isn't even that old because Intel has only launched Skylake Xeon E5 not long ago. And the different between S…

I suspect that the Xeon processors are running at a lower frequency under load compared to the EPYC ones.

That's just a hunch though.

Re: An EPYC NixOS Build Farm

#29

Post by praseodym »

ksec wrote: post #21
tmd83 wrote: post #2

This is quite surprising. I assumed they would get an EPYC server with higher core at similar price (due to the cost of the CPU) and get advantage in performance/dollar. I didn't expect EPYC to beat a similar physical core count intel server running at similar core count let alone by this much margin. Am I missing something here? I'm saying this as someone who really wants AMD to succeed and push the ecosystem forwar…

>They are effectively getting 35% extra performance 60% of the price that's just beyond impressive. Lets just ignore the price point for now because the memory capacity are different. But the performance, 24 Core AMD EPYC Zen is ~35% faster than 24 Core Intel Broadwell Xeon under the same frequency? Broadwell isn't even that old because Intel has only launched Skylake Xeon E5 not long ago. And the different between S…

Maybe their workloads were I/O bound and the EPYC server has faster SSDs?

Re: An EPYC NixOS Build Farm

#30

Post by chris_st »

aseipp wrote: post #27
chris_st wrote: post #9

I'm curious if anyone knows the test tool they're using here, in particular for lines like: $machine->waitForWindow("^Desktop "); It looks incredibly useful. Thanks!

It's our custom tool. NixOS has testing infrastructure built on top of QEMU, and you can use Perl scripts like this in order to automate aspects of testing inside that environment by talking to the QEMU monitor -- for example, you can not only wait for a window, but click a button in that window, then also wait for a service to start afterwords, for example. (If you were testing GUI functionality). Tests are written…

Thanks very much for this information (you and @grhmc).

PLEASE consider releasing some form of this tool, or at least documenting the snot out of it... it looks incredible. We'd really use this at work.

Post reply on HN