Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cheyenne Super Computer Auction

gsaauctions.gov

Post reply on HN
Print view View original

61–70 of 159 posts

Re: Cheyenne Super Computer Auction

#61

Post by ansible »

haunter wrote: post #59

Never understood why can you bid below the reserve price, or more like why the reserve price is hidden because the whole point that they (the seller) have a price in mind they are not willing to go below.

It is playing on the psychology of the bidders. You want the bidders to be invested, to want to win the auction. To compete to win the prize.

Also, consider this: if the reserve is too high, and no one bids on it, then everyone looking at it is going to wonder what it is really worth. If there are several other bidders, then that gives reassurance to the rest for the price they each are bidding.

Re: Cheyenne Super Computer Auction

#62

Post by michaelt »

jeffbee wrote: post #7

I wonder what the point of liquid cooling such a system was. Were they pressed for space?

The heat's going to be leaving the building in liquid-filled pipes, however you architect it. And with 1.7MW of peak power consumption, a nontrivial amount of liquid.

It's just a question of whether you want to add air and refrigerant into the mix.

It seems they're decommissioning it partly due to "faulty quick disconnects causing water spray" though, so an air cooling stage would have had its benefits...

Re: Cheyenne Super Computer Auction

#63

Post by latchkey »

mk_stjames wrote: post #37
observationist wrote: post #29

Earlier quoted context omitted.

In comparison, 18 A100 GPUs would have 5.6 petaflops and 1.4 TB vram, consuming 5.6 kw. The speed of processing and interconnect is orders of magnitude faster for an A100 cluster - 1 8 gpu pod server will cost around $200k, so around $600k more or less beats the supercomputer performance (price I'm searching seems wildly variable, please correct me if I'm wrong.)

The Cheyenne numbers are 5.34 petaflops of *FP64*. The 5.6PF you quote for 18 A100's would be in BF16. Not comparable. The A100 can only do 9.746 TFLOPS in FP64. So you would need 548 A100's to match the FP64 performance of the Cheyenne.

AMD MI300x is 163.4 TFLOPS in FP64.

33 of them, which would also have 6,336TB of memory.

I'll have way more than that in my next purchase order.

It is really fun to build a super computer.

Re: Cheyenne Super Computer Auction

#64

Post by latchkey »

queuebert wrote: post #6

I wonder who buys these. Crypto miners? My institution would make it nearly impossible to buy a secondhand supercomputer.

Crypto is no longer mined commercially with GPU type compute. When ETH switched to PoS, it decimated the entire GPU mining industry. It is no longer profitable. The only people doing it now are hobbyists.

Re: Cheyenne Super Computer Auction

#65

Post by ansible »

EvanAnderson wrote: post #48
vel0city wrote: post #46

Earlier quoted context omitted.

There's 8,064 E5-2697v4's in this. Those go on ebay for ~$50/ea. That's $400,000 of just CPUs to sell. If the winning bid is $100k, you spend $40k to move it out of there, another $10k warehousing it while selling everything on ebay, and you're still up $250k on the processors alone.

> That's $400,000 of just CPUs to sell. But do you crater the market for those CPUs? What's the demand for 2016-era Xeons and how much of their price comes down to supply?

I presume no one is building new motherboards for those processors either. While there is old stock laying around, you really need to run those systems close to as-is for them to be useful.

Re: Cheyenne Super Computer Auction

#66

Post by formerly_proven »

nickpsecurity wrote: post #55

My favorite part of SGI computers, like Altix and UV lines, was the NUMA memory with flexible interconnect. NUMA let you program a pile of CPU’s more like a single-node, multithreaded system. Then, the flexibility let you plug in CPU’s, graphics cards, or FPGA’s. That’s right into the low-latency, high-speed, memory bus. There was a company that made a card that connected AMD servers like that. I don’t know if such t…

Cray XT/SeaStar? iirc the interconnect ASIC pretends to be another peer CPU connected via HyperTransport. HPE Flex is similar, but works via QPI/UPI for Intel CPUs.

Re: Cheyenne Super Computer Auction

#67

Post by Tempest1981 »

h2odragon wrote: post #8

> the system is currently experiencing maintenance limitations due to faulty quick disconnects causing water spray. Given the expense and downtime associated with rectifying this issue in the last six months of operation, it's deemed more detrimental than the anticipated failure rate of compute nodes. Even the RAM has aged out... Very hard to justify running any of this; newer kit pays for itself in reduced power and…

Are you trying to discourage others from bidding, so you can swoop in and win the auction?

Re: Cheyenne Super Computer Auction

#68

Post by kube-system »

michaelt wrote: post #57
chasil wrote: post #25

Earlier quoted context omitted.

I can find a bunch of the E5-2697v4 CPUs on eBay in the $30-40 range. I wonder if there is a market for the SGI hardware.

So getting 8,064 of them for $3,085 - 38 cents per CPU - is great value for money!

Dump 8,064 old processors on eBay and you'll probably introduce some downwards price pressure.

Re: Cheyenne Super Computer Auction

#69

Post by jazzyjackson »

michaelt wrote: post #57
chasil wrote: post #25

Earlier quoted context omitted.

I can find a bunch of the E5-2697v4 CPUs on eBay in the $30-40 range. I wonder if there is a market for the SGI hardware.

So getting 8,064 of them for $3,085 - 38 cents per CPU - is great value for money!

this is basically "free grand piano" - not so free once you hire the movers and tuners

Re: Cheyenne Super Computer Auction

#70

Post by adgjlsfhk1 »

jeffbee wrote: post #38
martinpw wrote: post #36

Earlier quoted context omitted.

The supercomputer flops are FP64. The A100 stats you are using are FP16.

It's fine. We will simply run weather forecast in BF16 mode and hallucinate the weather.

weather forecasting is actually moving to reduced precision. none of the input data is known to more than a few digits, and it's a chaotic system so the numerical error is usually dominated by the modeling and spacial discretization error
Post reply on HN