Racking Mac Pros • Hacker News Board

Re: Racking Mac Pros

#161

Post by skuhn » Wed, May 06, 2015, 9:06 PM UTC

elif wrote: post #132

rsync wrote: post #109

I have two conflicting responses to what I am seeing here ... First, this is awesome. Just like I want to live in a world where people are paying picodollars for cloud storage[1], I also want to live in a world where a bunch of mac pro cylinders are racked up in a datacenter. Very cool. Second, this is complete silliness. I'm not going to go down the rabbithole of flops per dollar, but there is no way that you can't…

I'm not trying to lure you down the flops per dollar rabbithole... but when i was researching the mac pro before purchasing, putting together a computer with GPUs providing the same flop performance was something like 80-90% of the cost of the mac pro. the D700's are really good. This is just an anecdote based on my memory, so take with a grain of salt, but the comparison is probably not as bad as you are imagining.

This is what I found as well, and allude to in other comments in the thread. Particularly when you look at GPGPUs like the NVidia Tesla, they are generally pretty terrible when priced per-gflop because they live in a niche market that wants to handle finite element analysis and supercomputing tasks.

Image processing doesn't require double precision, so we don't need GPUs tuned for it, which means we can use Fire Pro's and similar workstation or server grade cards.

Re: Racking Mac Pros

#162

Post by anoother » Wed, May 06, 2015, 9:07 PM UTC

It would be interesting, once these have been in use for a while, to see some stats on the relative temperatures (+ fan speeds) of each machine within the enclosure.

I can imagine the dynamics of 4 machines scavenging air from a single chamber, with an opening on one end, will result in the machines nearer the warm aisle having to work harder to keep cool...

I also wonder what kind of ducting could be implemented to minimize this effect.

Anyway, a very cool project ending in what looks to be a fantastic end product. I wish I had the chance to work on something like this!

Re: Racking Mac Pros

#163

Post by sabujp » Wed, May 06, 2015, 9:08 PM UTC

apple's licensing terms force abominations such as these

Re: Racking Mac Pros

#164

Post by smackfu » Wed, May 06, 2015, 9:09 PM UTC

frugalmail wrote: post #133

This seems like a company destined to fail: 1) Massive premium for compute 2) They're at the mercy of Apple, a single completely unpredictable vendor. 3) Apple changes it's form-factors to the latest "design" way to frequently 4) Apple sucks to manage in mass

>Apple changes it's form-factors to the latest "design" way to frequently

Going on past history, I don't think they will have to worry about the Mac Pro being updated too often.

Re: Racking Mac Pros

#165

Post by listic » Wed, May 06, 2015, 9:09 PM UTC

angersock wrote: post #153

TD-Linux wrote: post #134

Given all of the effort spent to use Quartz's graphics operations, I was curious as to how they actually performed. I opened an account and tried out the upsampling, and was a bit disappointed. http://chen.imgix.net/rose.png?w=560 What other upsamplers look like: https://github.com/haasn/mpvhq-upscalers/blob/master/Rose.md Looking at the other operations available, I fail to see what is done better by Quartz than jus…

As I recall, the basic idea was for something lighter-weight than spinning up and spinning down imagemagick. Then again, one wonders why not just use FreeImage or something?

How does FreeImage compare to ImageMagick?

Re: Racking Mac Pros

#166

Post by skuhn » Wed, May 06, 2015, 9:10 PM UTC

listic wrote: post #129

zurn wrote: post #115

Earlier quoted context omitted.

> The power draw related to cooling for this rack versus a rack of conventional servers is about 1-5/th to 1/10th as much. Do you mean your Mac Pros dissipate 1/5 to 1/10th as much heat as other x86 server hardware, or is there there some other factor in play that makes your AC 5-10x more power efficient?

I understand "related to cooling" as Mac Pro's cooling in this setup is 5-10x more efficient.

Sorry, just some off-the-cuff math. We use Supermicro FatTwin systems for Linux stuff, and they run a lot of fans at much higher RPMs to maintain proper airflow relative to the Mac Pro design (which runs one fan at pretty low RPMs most of the time).

As a result, I'm calculating that the Mac Pros draw a lot less power for cooling purposes than the Linux systems due to their chassis design. However, serviceability and other factors are definitely superior on the Supermicro FatTwins.

Re: Racking Mac Pros

#167

Post by skuhn » Wed, May 06, 2015, 9:11 PM UTC

sajal83 wrote: post #122

skuhn wrote: post #71

Earlier quoted context omitted.

(I'm the datacenter manager at imgix, and I wrote this article) 1. Yeah, the OS X graphics pipeline is at the heart of our desire to use Macs in production. It's also pretty sweet to be able to prototype features in Quartz Composer, and use this whole ecosystem of tools that straight up don't exist on Linux. 2. I mentioned this elsewhere already, but it is actually a pretty good value. The chassis itself is not a ter…

Why not unassemble the cylinders and re-assemble into rectangle chasis? Im sure that would give you a more dense layout.. Sure it would void warranty and resale value.. but do you really care?

This actually came up during the design phase, and it was tempting. However, you'd have to figure out how to connect the boards together, and you'd have to figure out where to put heatsinks and where to direct airflow.

Since we were able to get the Pros to the point where they effectively occupy 1U, there wasn't really any incentive to doing a disassembly style integration. Maybe if Apple announces the next Mac Pro comes as a triangle.

To your other point about the warranty and re-sale: we do care, but only a little. I budget machines to have a usable lifespan of 3 years, but the reality is that Apple hardware historically has significant value on the used market for much longer than that. So if we can recoup $500-1000 per machine after 3 years of service, that would be great.

Re: Racking Mac Pros

#168

Post by TD-Linux » Wed, May 06, 2015, 9:11 PM UTC

smackfu wrote: post #147

TD-Linux wrote: post #134

Given all of the effort spent to use Quartz's graphics operations, I was curious as to how they actually performed. I opened an account and tried out the upsampling, and was a bit disappointed. http://chen.imgix.net/rose.png?w=560 What other upsamplers look like: https://github.com/haasn/mpvhq-upscalers/blob/master/Rose.md Looking at the other operations available, I fail to see what is done better by Quartz than jus…

I assume 99% of users are using downsampling, to get thumbnails.

CHROME ON MAC USERS: Use the imgur links instead of the 0x0 links, some users seem to be reporting crashes related to TLS.

The downsampling also isn't that great.

Original image: https://raw.githubusercontent.com/haasn/cms/master/rings_lg_...

Downsampled with imgix: http://chen.imgix.net/rings_lg_orig.png?w=400

Downsampled with imagemagick: https://0x0.st/1-.png http://i.imgur.com/Nvl7tAm.png

Downsampled with imagemagick, gamma correct: https://0x0.st/1i.png http://i.imgur.com/Hrm4COb.png

Note how the luminance becomes square in the center (step back a bit if you can't see it), and also the edge pixels on the imgix version.

Re: Racking Mac Pros

#169

Post by angersock » Wed, May 06, 2015, 9:12 PM UTC

listic wrote: post #165

angersock wrote: post #153

Earlier quoted context omitted.

As I recall, the basic idea was for something lighter-weight than spinning up and spinning down imagemagick. Then again, one wonders why not just use FreeImage or something?

How does FreeImage compare to ImageMagick?

It handles image loading, unloading, and basic manipulation options.

I imagine, especially if the traffic is mainly for downsampling, it'd be sufficient. If it's not, then writing some custom code to do the image transforms on a GPU and bring them back shouldn't be that gnarly--and if you can afford to stick a shitton of macs in a data center, you can afford a graphics programmer to get that done.

Re: Racking Mac Pros

#170

Post by skuhn » Wed, May 06, 2015, 9:12 PM UTC

angersock wrote: post #151

tlrobinson wrote: post #136

Earlier quoted context omitted.

Core Image: https://developer.apple.com/library/mac/documentation/Graphi... http://en.m.wikipedia.org/wiki/Core_Image I'm not super familiar with it or the competition, but I assume this is what they're talking about.

So, it's basically the MESA Intel graphics pipeline? EDIT: For the downvoters and the unclear, the relevant bit talks about compiling exactly the instructions needed to change the image. As I understand it, this JIT recompilation of pixel shaders is effectively what was implemented in the mesa drivers for Intel chipsets.

Compiling the shaders is a big win, since it allows us to do almost all operations in one pass rather than multiple passes. The service is intended to function on-demand and in real time, so latency matters a lot.