121–130 of 285 posts
Earlier quoted context omitted.
Why weren't crypto gpus running at 100%?
Most crypto mining on GPUs would use 100% of memory bandwidth, but only a fraction of the compute available. This is a consequence of ASIC resistance of their mining algorithms -- custom silicon can only offer a modest benefit over GPUs if the hard part is memory bandwidth.
Earlier quoted context omitted.
GPGPU? General Purpose GPU? If embarrassingly parallel CPU algorithms weren't offloaded to the GPU previously, why would the A100/H100 price drop make a difference? We had cheap GPU in the past and we still left plenty of performance on the table with CPU programs because they were easier to build. Is the idea that previously maintaining GPU programs was expensive whereas now AI makes it cheap? If so, I could buy tha…
GPGPU programming has become significantly easier now and the payoff is bigger (better hardware), due to the immense investment in this due to ML/AI.
I half expect Nvidia to have buyback contacts like Ferrari with the larger customers to prevent a price crash when they all upgrade and to keep them scarce. I hope not though, perhaps I can pick up a H100 in a few years if they get sold on the open market.
The only reason a datacenter would ditch their H100 is if it becomes uneconomical to run them, with newer silicon providing much more power efficiency. When that happens they'll look like a used V100 looks today: horribly inefficient, lacking modern data types and engines, requiring screaming server fans with weird adapters to not melt, way beyond end of life in terms of cuda support. Almost completely damn useless u…
Isn't it worth exactly what you can sell it for. a few ways to do this. slow and awkward, best market match: The auction. sell to highest bidder. faster and more customer friendly but poor market match until a lot of units sold: The store. guess price, adjust up or down to reach sell frequency desired. fast and good market match but takes a knowledgeable customer base: The reverse auction. Start with price too high l…
>Start with price too high lower it over time until it sells.
These are the same strategy.
Earlier quoted context omitted.
crypto gpus didnt run at 100% power draw so the strain wasn't that massive
I bought a RTX 3070 off a miner when Eth went to proof of stake. It was clean, cheap, and is still going strong for daily gaming. In its working life it was undervolted and probably cooled better than in my rig.
It's wild to think that the system now is worth at least as much as I paid for it then if not much more than that. I saw a similar one going for $2500.
Earlier quoted context omitted.
There's going to be a golden age of GPGPU compute in the next few years once A100/H100 are fully obsolete for running frontier models efficiently and the price plummets It will be perfect for stuff like GPU-accelerated query engines, "classical ML" and every other CPU-based workload that could conceivably be offloaded to GPU
GPGPU? General Purpose GPU? If embarrassingly parallel CPU algorithms weren't offloaded to the GPU previously, why would the A100/H100 price drop make a difference? We had cheap GPU in the past and we still left plenty of performance on the table with CPU programs because they were easier to build. Is the idea that previously maintaining GPU programs was expensive whereas now AI makes it cheap? If so, I could buy tha…
"Secondary market data shows moderately-used 2 to 3 year old GPUs trading at 50% to 70% of new pricing under normal conditions." That's for good NVidia H100 units.[1] There's a shortage of those. That seems to be the price after removal, cleaning, testing and refurbishing. Raw units removed from a shutdown will not be as valuable. H100 units are available on eBay, but multiple sellers are using the same picture of a…
Who is buying these at 70% of new pricing given the sky high likelihood of them being shot? Maybe it's safe to buy from small labs that went under quickly, but I can't imagine a cluster that has been operating near its thermal limits for a couple years fetching that kind of resale.
Earlier quoted context omitted.
How does this work? Is the OEM giving the cloud provider a big discount or is the cloud provider giving you a teaser rate to lock up your business.
The list price of cloud is 10x higher than the price of hardware itself so that leaves room for some discounting.
Where folks (often) get lazy is the resulting math over what the real bean-counters care about (but are too lazy to check often).
In a past life, I worked on costing models for a Cable/Fiber contract house, to help the company decide 'what was profitable to keep in house' versus 'what do we subcontract' (sometimes that could even mean we just 'rented' a machine and had a qualified operator using it, based on that employee's hourly rate and expected L2R for taxes... so many spreadsheets...)
And from from my 'I don't know all the factors for this but I've seen how people screw up the big ones' view (and frankly, I'm guessing a lot of us have seen and dealt with the same category of 'bad math' around outsourcing IT work...)
An on-prem data center means:
- You need to account for electricity costs - i.e. CA vs midwest electric rates.
- cooling and power backup capability - Smaller factor but real
- personnel cost - e.x. there's probably cases where a smaller org could be better off with 'on-site' server admins that have other roles based on local wages. Kinda case specfic but it's a case.
- whatever the 'space' holding the stuff costs
- Sardonic take :Hey, let's have another unused meeting room instead! (e.x. In the case of on-prem shops that simply fled to AWS in their migration from VMware)
- the cost of licensing whatever is running
- In defense of this, In one of my earliest IT lives, AWS handling the Oracle licensing for a DB was a *huge* win as far as making it as easy as possible to ensure whatever was going on we couldn't have the Oracle licensing folks 'ding' us on whatever infraction occurred between reviews (that could not be understood by the majority of the company, often including the accused. I was never guilty but I saw it happen to others.)
- OTOH I know lots of folks who just want to be lazy about what they have to document.
Frankly, the orgs that 'might' need that either have their arch set up where they are in cloud, or they are onprem but can scale to cloud if needed in interim.
One data point: 512GB Mac Studios are selling on ebay for double what they were selling for new at the beginning of the year.