211–220 of 265 posts
Earlier quoted context omitted.
A Strix Halo with 128GB unified memory is less than $2k and the more suitable alternative to a mac. I'm pretty happy with my device (Bosgame M5).
the macs outperform it and I figure it's a better general purpose computer than strix halo. if budget is a problem, then a strix halo is a decent alternative.
Earlier quoted context omitted.
And if you don't want to buy a Mac? A 80 GB NVidia GPU costs $10,000K (equivalent to 30 years of ChatGPT Plus subscription) and will probably be obsolete in 5-7 years anyway. What are my options if I want a decent coding agent at a reasonable price?
What are my options if I want a decent coding agent at a reasonable price? I'd even come from another angle.. What are my options if I want a decent coding agent, on the level of what Claude does at any given price? Let's say few tens of thousands of dollars? I've had a limited look at what's available to be run locally and nothing is on par.
If you want to spend big bucks get h200 141 GB but honestly RTX 6000 pro is good enough till you know what you want. Workstation edition is good. It takes care of cooling etc.
Tbh even better is to just get model through cloud. If you want you can rent GPU. Then see if it’s what you want.
Earlier quoted context omitted.
GPUs are not going obsolete anytime soon. the nvidia p40/p100 launched in 2016, 10 years ago and is popular in the local space. My first set of GPUs were a bunch of P40s from 3 years ago for $150 a piece. They at one point went up all the way to $450, but price is now down to $200 range. I think I have gotten my value from those and I suspect I'll still have them crunching out tokens for at least 3 more years. They s…
Indeed, the point is that it's going for 150$
Earlier quoted context omitted.
I'm able to run the Unsloth quants on an ancient dual socket Xeon 1U server I keep around for homelab stuff. It has 8 DDR3 channels, which gives me about as much memory bandwidth as two channels of DDR5 :-/ But 16 sockets and cheaper prices. So it has 256gb in it right now. I have to run the minimum size Unsloth quant for the largest open weight models. They definitely feel a bit dazed. This machine can support up to…
> Power consumption That’a the thing, at the end of it all power consumption will matter more for the end-user who doesn’t have money to burn away, because I suspect that power-consumption will, in the majority of cases, exceed the price of the HW itself in a matter of just a few months of intense use, let’s say a year.
If I were to try to purpose build a rig for it, I would get an engineering sample Epyc/motherboard/ram combo from Aliexpress with 12 channels of DDR5 and as few cores as allowed me to still use all the memory bandwidth, and I'd run it at the lowest possible power and voltage settings with aggressive ram timings. A system like that can draw 1/3 of what my scrap rig draws, at full load. And has similar memory bandwidth to a high end Mac or GPU allowing it to crank out 5 - 10 Tokens / s on the largest models, which works out to 1/3 of a penny to 2/3 of a penny per token. But either way, Epyc or Mac is going to set you back $10k or more. Hopefully in a few years when they are scrap though...
Earlier quoted context omitted.
What are my options if I want a decent coding agent at a reasonable price? I'd even come from another angle.. What are my options if I want a decent coding agent, on the level of what Claude does at any given price? Let's say few tens of thousands of dollars? I've had a limited look at what's available to be run locally and nothing is on par.
Does not exist AFAIK. Even other labs struggle with Claude level performance in real world task. My experience is that no open model is close. You can get RTX 6000 Pro Blackwell (Max-Q is better for power is half). I have heard good things about Qwen3 coder next but I could not get tool calling to be high performance but it’s likely to be pebkac. If you want to spend big bucks get h200 141 GB but honestly RTX 6000 pr…
Earlier quoted context omitted.
I guess that means they are only soft deleting.
The delete screen says they delete your data except where necessary by law for them to keep. Would love to know what percentage actually is deleted.
Earlier quoted context omitted.
They specifically say you cannot reuse the same email. I also tried after deleting and it said this email has been deleted.
Huh? That seems weird? So if I no longer want their services for any period of time, they no longer want me as a customer for any reason? What other business work like that?
Earlier quoted context omitted.
A Strix Halo with 128GB unified memory is less than $2k and the more suitable alternative to a mac. I'm pretty happy with my device (Bosgame M5).
A Strix Halo with 128GB unified memory is less than $2k Where did you get that price? Wherever I looked it's around 3k euros which is around $3.5k
Earlier quoted context omitted.
"Only when the tide goes out do you discover who's been swimming naked." -- Warren Buffet The original context was very different, about financial markets, but I've been thinking about it a lot the past 12 months. There's a lot of cowards in high places in tech, surprisingly cowardly people. Or they have sold out their principles to be friends with terrible people, which is also a form of cowardice. Hard to say which…
I think we merely have a system where the best people are selfless and poor and the worse people are rich and in charge. It makes sense; we have a system that rewards immoral behavior so we shouldn't be surprised that immoral people have made it to the top.