Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cheyenne Super Computer Auction

gsaauctions.gov

Post reply on HN
Print view View original

81–90 of 159 posts

Re: Cheyenne Super Computer Auction

#81

Post by bombcar »

HeyLaughingBoy wrote: post #60
humansareok1 wrote: post #54

Earlier quoted context omitted.

Auctions by default almost always undercut the actual market value so no not really?

My answer was a bit tongue-in-cheek, but the reality is that it depends on what you mean by "market value." For the market of the auction, the selling price is the actual market value. Likewise, it's typically not too far off the value of the item in the wider market, assuming you are comparing it to a similar item in similar condition. The problem is that for most items purchased at auction, there's no similar item,…

It's exactly in this "now vs later" that resellers and other brokers sit. If they know that X will sell for $Y "eventually" and how long that eventually is, they can work out how much they can pay for it now and still come out ahead.

Cars are very liquid and move quickly, so the now vs later price is close; weird things that nobody has heard of (but when they need it, they need it NOW) will have a much wider variance.

Re: Cheyenne Super Computer Auction

#83

Post by Locutus_ »

chasil wrote: post #25
Isamu wrote: post #4

>Components of the Cheyenne Supercomputer Installed Configuration: SGI ICE™ XA. E-Cells: 14 units weighing 1500 lbs. each. E-Racks: 28 units, all water-cooled Nodes: 4,032 dual socket units configured as quad-node blades Processors: 8,064 units of E5-2697v4 (18-core, 2.3 GHz base frequency, Turbo up to 3.6GHz, 145W TDP) Total Cores: 145,152 Memory: DDR4-2400 ECC single-rank, 64 GB per node, with 3 High Memory E-Cells…

I can find a bunch of the E5-2697v4 CPUs on eBay in the $30-40 range. I wonder if there is a market for the SGI hardware.

There is, but really only for the MIPS hardware.

Re: Cheyenne Super Computer Auction

#84

Post by gh02t »

bombcar wrote: post #78
gh02t wrote: post #23

Earlier quoted context omitted.

I can imagine it, some people on there are ridiculous, but yeah in my experience these supercomputer nodes are a lot more integrated/proprietary than most standard server hardware. It's not straightforward to just boot one up without all the support infrastructure. I'd assume they'd mostly be torn down and parted out.

You might be surprised - because they're pretty custom they are often "more open" than you might expect; as long as you have the connectors you can often get things running something . Sometimes they have bog-standard features present on the boards, just not enabled, etc. It's the commoditized blade servers, etc that are stripped down to what they need to run and nothing more.

Oh I'm speaking from experience with the SGI supercomputer blades. They're pretty wacky, 4x independent, dual cpu boards per blade and all sorts of weird connectors and cooling and management interfaces. Custom, centralized liquid cooling that requires a separate dedicated cooling rack unit and heat exchanger, funky power delivery with 3 phase, odd networking topologies, highly integrated cluster management software to run them etc. I'm not sure if they have any sort of software locks on top of that, but I would bet they do and presumably NCAR wipes all of them so you likely won't have the software/licenses.

I dug up a link to some of the technical documentation https://irix7.com/techpubs/007-6399-001.pdf . Probably someone can get it working, but I imagine whoever is going to go through the hassle of buying this whole many-ton supercomputer is planning to just strip it down and sell the parts.

Re: Cheyenne Super Computer Auction

#85

Post by humansareok1 »

organsnyder wrote: post #58
humansareok1 wrote: post #54

Earlier quoted context omitted.

Auctions by default almost always undercut the actual market value so no not really?

Isn't the winning bid the actual market value, by definition?

I think Auctions exist explicitly to potentially buy or sell an Item with a delta on it's Market Value? I.e. Buyers want the chance to buy below and sellers to sell above. Neither really wants to engage in the transaction at all in the reverse situation or even in the "Market Value" case. You would just make a direct sale and avoid the hassle of an auction.

Re: Cheyenne Super Computer Auction

#86

Post by pantalaimon »

ansible wrote: post #65
EvanAnderson wrote: post #48

Earlier quoted context omitted.

> That's $400,000 of just CPUs to sell. But do you crater the market for those CPUs? What's the demand for 2016-era Xeons and how much of their price comes down to supply?

I presume no one is building new motherboards for those processors either. While there is old stock laying around, you really need to run those systems close to as-is for them to be useful.

> I presume no one is building new motherboards for those processors either

That's actually far from the truth, LGA2011 is quite popular as a budged gaming system precicely because CPUs are so cheap on the 2nd hand market.

https://aliexpress.com/w/wholesale-X99.html

Re: Cheyenne Super Computer Auction

#87

Post by bombcar »

gh02t wrote: post #84
bombcar wrote: post #78

Earlier quoted context omitted.

You might be surprised - because they're pretty custom they are often "more open" than you might expect; as long as you have the connectors you can often get things running something . Sometimes they have bog-standard features present on the boards, just not enabled, etc. It's the commoditized blade servers, etc that are stripped down to what they need to run and nothing more.

Oh I'm speaking from experience with the SGI supercomputer blades. They're pretty wacky, 4x independent, dual cpu boards per blade and all sorts of weird connectors and cooling and management interfaces. Custom, centralized liquid cooling that requires a separate dedicated cooling rack unit and heat exchanger, funky power delivery with 3 phase, odd networking topologies, highly integrated cluster management software…

Yeah the licensing is often the stumbling block, unless you can just run some bog-standard linux on it. It sounds like this might be custom enough that it would be difficult (but I daresay we'll see a post in 5 years from someone getting part of it running after finding it on the side of the road).

Re: Cheyenne Super Computer Auction

#88

Post by tombert »

Man, if I had the space, the money, and the means of powering it I would bid on this immediately. It's so damn cool and will likely end up selling for a lot less than its worth due to its size.

I've always been fascinated by the supercomputer space, in no small part because I've been sadly somewhat removed from it; the SGI and Cray machines are a bit before my time, but I've always looked back in wonder, thinking of how cool they might have been to play with back in the 80s and 90s.

The closest I get to that now is occasionally getting to spin up some kind of HPC cluster on a cloud provider, which is fun in its own right, but I don't know, there's just something insanely cool about the giant racks of servers whose sole purpose is to crunch numbers [1].

[1] To the pendants, I know all computers' job is to crunch numbers in some capacity, but a lot of computers and their respective operating systems like to pretend that they don't.

Re: Cheyenne Super Computer Auction

#89

Post by mk_stjames »

latchkey wrote: post #63
mk_stjames wrote: post #37

Earlier quoted context omitted.

The Cheyenne numbers are 5.34 petaflops of *FP64*. The 5.6PF you quote for 18 A100's would be in BF16. Not comparable. The A100 can only do 9.746 TFLOPS in FP64. So you would need 548 A100's to match the FP64 performance of the Cheyenne.

AMD MI300x is 163.4 TFLOPS in FP64. 33 of them, which would also have 6,336TB of memory. I'll have way more than that in my next purchase order. It is really fun to build a super computer.

I'm an amateur, but I have code that I think could probably dispatch threads pretty efficiently on the Cheyenne thru it's management system simply because it's all xeons distributed. If I can run it on my personal 80-core cluster, I could get it to run on Cheyenne back then.

But hitting the roofline on those AMD GPGPU's? I'd probably get nowhere fucking close.

That is the thing that Cheyenne was built for. People doing CFD research with x86 code that was already nicely parallelized via OpenMPI or whathaveyou.

Re: Cheyenne Super Computer Auction

#90

Post by Animats »

It's really hard to find a home for large, old, high-maintenance technology. What do you do with a locomotive, or a Linotype? They need a support facility and staff to be more than scrap. So they're really cheap when available.

The Pacific Locomotive Association is an organization with that problem. About 20 locomotives, stored at Brightside near Sunol. They've been able to get about half of them working. It's all volunteer. Jobs that took days in major shops take years in a volunteer operation.

Post reply on HN