Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Racking Mac Pros

photos.imgix.com

Post reply on HN
Print view View original

201–210 of 331 posts

Re: Racking Mac Pros

#201

Post by xaduha »

grecy wrote: post #192
rsync wrote: post #109

I have two conflicting responses to what I am seeing here ... First, this is awesome. Just like I want to live in a world where people are paying picodollars for cloud storage[1], I also want to live in a world where a bunch of mac pro cylinders are racked up in a datacenter. Very cool. Second, this is complete silliness. I'm not going to go down the rabbithole of flops per dollar, but there is no way that you can't…

> there is no way that you can't build a hackintosh Have you ever personally run a Hackintosh, full-time for a prolonged period of time? It's anecdotal, but I can assure that once you're used to how OS X and the Apple hardware work together and never, ever, ever crash, using a Hackintosh is an exercise in frustration. I had one of the known-best Hackintosh configurations in existence, and it didn't hold a candle to t…

> Have you ever personally run a Hackintosh, full-time for a prolonged period of time?

I did. But now I'm running Yosemite under KVM, VT-d motherboard, dedicated videocard and USB3 hub.

You can get to a point where "it just works".

Re: Racking Mac Pros

#202

Post by jpgvm »

grecy wrote: post #192
rsync wrote: post #109

I have two conflicting responses to what I am seeing here ... First, this is awesome. Just like I want to live in a world where people are paying picodollars for cloud storage[1], I also want to live in a world where a bunch of mac pro cylinders are racked up in a datacenter. Very cool. Second, this is complete silliness. I'm not going to go down the rabbithole of flops per dollar, but there is no way that you can't…

> there is no way that you can't build a hackintosh Have you ever personally run a Hackintosh, full-time for a prolonged period of time? It's anecdotal, but I can assure that once you're used to how OS X and the Apple hardware work together and never, ever, ever crash, using a Hackintosh is an exercise in frustration. I had one of the known-best Hackintosh configurations in existence, and it didn't hold a candle to t…

I have been using a Hackintosh as my primary rig going on 4+ years now. It can be frustrating if you are trying to use the very latest hardware but I find the small issues a decent tradeoff.

It's not a matter of cheaper for me, but a matter of fitting my needs. I don't want to run AMD graphics cards, I need PCI-E, I want lots of internal storage, I want really high single threaded CPU performance.

I can't buy that from Apple in a desktop form-factor. So I have my Hackintosh.

That being said, I don't disagree that Apple hardware is nice. I have a rMBP 13 and intend on replacing it with a newer model Apple notebook soon.

Re: Racking Mac Pros

#203

Post by skuhn »

ansible wrote: post #123
skuhn wrote: post #105

Earlier quoted context omitted.

The Tesla K80 didn't exist when I started this project, but to do some quick math: K80 gflop/s: 8740 2x FirePro D500 gflop/s: 3500 K80 runs about $4900 a card, whereas the entire Mac Pro (list price) is $4000. So it's 2.5x the performance at easily 2x the cost if not more. You're right that there is a cost advantage to going with commodity server hardware, but I don't think it's as great as most people think in this…

2x FirePro D500 gflop/s: 3500 That 3500 gflop/s for the D700? It is instead 2200 for the D500. http://www.amd.com/en-gb/solutions/workstations/d-series K80 runs about $4900 a card, whereas the entire Mac Pro (list price) is $4000. So it's 2.5x the performance at easily 2x the cost if not more. The 6GB VRAM version with the D700 costs another $600 USD each. The K80 has 12GB VRAM per GPU (24GB total per card). If your…

Sorry, I do have this evaluation in a spreadsheet somewhere (except against the Tesla K20, K80 wasn't out then), but I just quickly looked up the Mac Pro specs. We do use the D500, so I should have quoted those gflops. There is a benefit to off-the-shelf GPUs, but I don't see it as a make-or-break kind of situation for imgix right now.

I agree that some day in the future, it does seem like it will make sense to bite the bullet and rewrite for Linux. It probably won't solely come down to a cost rationale though, because there are a TON of business risks involved in hitting pause on new features (or doubling team size, or some combination thereof).

Fundamentally I don't believe in doing large projects that have a best case scenario of going unnoticed by your customers (because the external behavior has not changed, unless you screwed up), unless you absolutely have to.

The real reason to migrate to Linux would have to be a combination of at least three things:

  1. Better hardware, in terms of functionality or price/performance
  2. Lower operational overhead
  3. The ability to support features or operations that we can't do any other way
Much more likely, we would adopt a hybrid approach where we still use OS X for certain things and Linux for other things.

Re: Racking Mac Pros

#204

Post by skuhn »

sabujp wrote: post #144

8 mp's in a 7u : https://www.facebook.com/pages/Rack-Your-Mini/10374582632541...

We use MK1's Mac Mini shelf (as seen at http://photos.imgix.com/building-a-graphics-card-for-the-int...), which is pictured at that page. I just wasn't in love with their Mac Pro shelf design, so we went a different direction.

Keep in mind also that 8 Pros in 7U = 48 in a 44U space. So it's a pretty similar density, but I don't think it is as ideal in terms of airflow. Instead, it's more ideal for working on the systems individually (such as in a colocation environment), but that isn't a particular concern of ours.

Re: Racking Mac Pros

#205

Post by skuhn »

derefr wrote: post #59

Apropos to nothing, but when you stuff a bunch of Mac Pros in a box, they begin to look like enlarged vacuum tubes/capacitors. I can almost imagine them being "screwed into" the rack chassis.

You do have to employ a bit of a twisting motion to remove them, since they have some gasketing in place. I wanted to add a dry ice smoke machine and blue LEDs, but alas...

Re: Racking Mac Pros

#206

Post by skuhn »

mschuster91 wrote: post #85

How do you remote provision OS X? I mean, how do you get to the boot menu to choose network boof? With the crashcart?

The crash cart is the method of last resort, and it's come into play a fair amount as we were figuring out how to do this.

The better solution is to have a NetRestore server on the network, and configure the Macs to boot from a particular server with the bless and nvram commands. Then on the server, you control if the image gets served or not based on some external logic (in my case, an attribute in my machine database).

At the moment, NetRestore is running on an OS X Server machine hanging out on the network, but integrating it with our existing Linux netboot environment is on my to-do list.

Re: Racking Mac Pros

#207

Post by grecy »

xaduha wrote: post #201
grecy wrote: post #192

Earlier quoted context omitted.

> there is no way that you can't build a hackintosh Have you ever personally run a Hackintosh, full-time for a prolonged period of time? It's anecdotal, but I can assure that once you're used to how OS X and the Apple hardware work together and never, ever, ever crash, using a Hackintosh is an exercise in frustration. I had one of the known-best Hackintosh configurations in existence, and it didn't hold a candle to t…

> Have you ever personally run a Hackintosh, full-time for a prolonged period of time? I did. But now I'm running Yosemite under KVM, VT-d motherboard, dedicated videocard and USB3 hub. You can get to a point where "it just works".

> You can get to a point where "it just works".

For months and months on end of heavy usage without a single restart or issue?

EDIT: to expand a little - I was developing/compiling all day long on my ~2008 MBP with it plugged into an external monitor, network, mouse, kb. I'd close the lid and walk home with it, then watch movies, torrent, develop some more, surf etc. Close lid, and repeat for months on end. The only time I ever restarted was for OS updates, I never had a single app even crash in ~2 years of doing that.

My hackintosh (and the windows 7&8 HP machines here at work) don't hold a candle to that.

Re: Racking Mac Pros

#208

Post by dba7dba »

bane wrote: post #29

It's really kind of mind-boggling that Apple makes and sells the Pro, which can be upgraded to a really nice high performance GPU workstation, but then doesn't sell the same hardware in rack mountable forms for clusterable computing. I'm sure they've performed some kind of market analysis for this, but there's enough differences between OSX and Linux solutions that for people who use HPC solutions (a growing market)…

Having worked with some rack mountable apple servers, I have a feeling they either don't care about having their hardware installed in the data center or don't know how to do it well. Believe it's the case of don't care.

I personally felt it was a disgrace to see Apple logo on the apple's rack mount servers.

Considering how little rack mounted equipment is replaced versus consumer hardware, I can see why.

Re: Racking Mac Pros

#209

Post by boyter »

Having just finished rolling out a largish scale Thumbor implementation has anyone compared it to Imgix?

The feature set I required is served by both equally, so it comes down to performance/ddos prevention/cost for me mostly. I am unlikely to modify what I have just done since it is working fine, but for the future would love to know if anyone has experience with this.

Post reply on HN