Volta (and Pascal, which I'm using) should still be supported with driver 580 as long as you don't use the open modules, and you can use up to cuda 12.9 and cudnn 9.10.2. No need to limit yourself to an old kernel.
I put a datacenter GPU in my gaming PC
61–70 of 199 posts
Re: I put a datacenter GPU in my gaming PC
#62Earlier quoted context omitted.
Where do you think llms learned to write that way?
Because their custom training data contains an emphasis on such verbiage. It doesn't come from the God-knows-how-many TB of web content the model is pre-trained on. There, such phrasing is only a drop in the sea. But the "yes, you're right" phrases, the em dash, etc., come from the later stage, for which content is created according to some (probably overprecise) guidelines.
Right. The overuse of "genuinely" most of all. Seems like they put Claude through a few good rounds of training to always answer questions about its consciousness, thoughts, etc., with something about how it's "genuinely unsure," and as a result, the model learned to use "genuinely" as an intensifier in all sorts of inappropriate contexts.
Re: I put a datacenter GPU in my gaming PC
#63A little bit of local copium but neat read. Isn't a rasbpi with 16gb of RAM $300 now?
The latest Raspberry Pi 5 has one 32-bit channel (2x 16-bit subchannels) of LPDDR4X-4267 SDRAM giving 17.1GB/s of bandwidth, 52x less than this GPU. Never mind lacking the CUDA and Tensor cores, so the FP16 performance is 102x less (307 GFLOPS vs 31.4 TFLOPS). So for £200, there's absolutely no comparison for this specific use-case.
Yeah thats what I'm saying. How is it so cheap????
Re: I put a datacenter GPU in my gaming PC
#64> And yes, if you want the absolute best, Opus 4.8 exists. It also costs more per 20 minutes of heavy use than I paid for this entire GPU and adapter setup combined. But the gap is shockingly small. I don't think this is a fair characterization of the situation. I use frontier models via API pre-paid tokens every single day, and I can barely rack up $100 per month . The fact that we figured out how to burn double thi…
Claude is something like $35 per million tokens. If I was using API pricing I could trivially spend $100 in a single hour long coding session, with /fast turned on in about 10 minutes. Not sure how you guys are using it.
coding is the easy part of using claude
Re: I put a datacenter GPU in my gaming PC
#65> And yes, if you want the absolute best, Opus 4.8 exists. It also costs more per 20 minutes of heavy use than I paid for this entire GPU and adapter setup combined. But the gap is shockingly small. I don't think this is a fair characterization of the situation. I use frontier models via API pre-paid tokens every single day, and I can barely rack up $100 per month . The fact that we figured out how to burn double thi…
Claude is something like $35 per million tokens. If I was using API pricing I could trivially spend $100 in a single hour long coding session, with /fast turned on in about 10 minutes. Not sure how you guys are using it.
Opus is normally $5 per mtok, no idea why anyone would use /fast if they were at all concerned about price. ($5 is still pricy though tbh)
Re: I put a datacenter GPU in my gaming PC
#66The real question: did your local LLM write this post?
There are many tells aren't there? There was clearly hard human work and experimentation here, but it's a shame the OP let AI do chunks of the writing. Once you see it, it's much harder to take the post seriously.
I disagree. Not everyone has a good writing style. In those instances I think it is fair to default to llm recommendation. We may be allergic to it, but we saw one formulaic response too many ( though admittedly it does raise a question of whether HN was the intended audience for it ).
In any event, not all of us have a unique writing style worth preserving just like not all of us can write clear and clean code. Just saying.
Re: I put a datacenter GPU in my gaming PC
#67Earlier quoted context omitted.
> Consider this: in 5-10 years, the trillions spent on AI data centers will likewise be sold for scrap most likely. That's how short the runway is for OpenAI and Anthropic to recover that investment. Even more interesting: it'll devalue all of SaaS and the entire US tech sector. We might have just shot our most valuable non-AI tech products in the foot.
How so? I understand that flooding the market with physical goods will reduce prices and thus profits. But how would that also reduce the nonphysical SAAS stuff?
> But how would that also reduce the nonphysical SAAS stuff?
The resulting economic crash will affect everyone, we're (IMHO) looking towards a dotcom-bust level wipeout. And many SaaS and other companies run asset-lean (i.e. they have no server hardware because that's all cloud, no real estate because it's all either wework or conventionally rented), margin-lean (the VC business model requires that, as the basic recipe is to achieve market domination by burning cash) and cash-lean (often enough, it's less than a quarter of expenses on the bank accounts).
All that "lean-ness" looks great on an investor's quarterly release sheet: no massive amounts of wealth tied up in assets and no cash sitting around on bank accounts that could be released towards investors as dividends or, if it comes from third parties, costs the company interest... but it prevents resiliency against crises.
Re: I put a datacenter GPU in my gaming PC
#68Some resell group is going to have to make this easier. The shear amount of these cards otherwise heading towards the landfill is staggering. That is if Big Tech don't destroy them to prevent model weights from leaking.
I genuinely hope that is the case. The market is absolutely bananas now. I actually now own devices that went up in 'value' since purchase. This is not normal ( and a little scary ). This, on the other hand, is an invitation to properly recycle otherwise unwanted hardware.
Re: I put a datacenter GPU in my gaming PC
#69Some resell group is going to have to make this easier. The shear amount of these cards otherwise heading towards the landfill is staggering. That is if Big Tech don't destroy them to prevent model weights from leaking.
Isn't this the same thing with 32 GB already on a PCIe socket? https://www.ebay.com/itm/166850431555
kinda, they put that on a PCIe socket, but it's passive. Meaning no fan. If you try inference on that it overheats in 1 minute unless you have it inside a server case.
Re: I put a datacenter GPU in my gaming PC
#70Earlier quoted context omitted.
Where do you think llms learned to write that way?
You can also look at past posts by the same author (before LLM usage proliferated) if you’re curious. The project is still very cool, but it’s a little less enjoyable to read when everything sounds the same. It would be just as annoying for people to manually write in a corporate/marketing style, because humanity is what makes the small web interesting. https://blog.tymscar.com/posts/privategithubcicd/
This, setting aside the llm issue, it is dealing with hardware in ways that -- one would think - would be celebrated on HN of all places. But we focus on presentation.