301–310 of 346 posts
Earlier quoted context omitted.
What they meant is buying two whole separate computers.
I understand. It is still unclear if you can get 128GB vram for $3k.
>Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage.
Even if $3k is only the starting price, it doesn't sound like spending more buys you more memory.
Random observation 1: I was running DeepSeek yesterday on my Linux with a RTX 4090 and I noticed that the models should fit into VRAM, which is 24GB. Or they are simply slow. So the Apple shared memory architecture has an advantage here. A 192GB Mx Ultra can load and process large models efficiently. Random observation 2: It's time to cancel the OpenAI subscription.
Earlier quoted context omitted.
I understand. It is still unclear if you can get 128GB vram for $3k.
Well, I mean, the press release is pretty unambiguous. >Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage. Even if $3k is only the starting price, it doesn't sound like spending more buys you more memory.
Earlier quoted context omitted.
Well, I mean, the press release is pretty unambiguous. >Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage. Even if $3k is only the starting price, it doesn't sound like spending more buys you more memory.
Ok, but it is not clear what kind of RAM is that, how many memory channels, etc. If the goal is to have just 128GB of some ram, then it could be achieved by paying few $100.
Earlier quoted context omitted.
Ok, but it is not clear what kind of RAM is that, how many memory channels, etc. If the goal is to have just 128GB of some ram, then it could be achieved by paying few $100.
Fine, but at that point you're arguing about the concept of the product. It's billed as a computer for AI and you're saying that it might not be more suitable for AI than a regular PC.
Earlier quoted context omitted.
Is there a tool that can automate chaining like that?
Aider has an architect mode where it asks one model to plan out the changes and another to actually write the code.
Alex (https://alexcodes.app) also does this now btw.
Earlier quoted context omitted.
Not everyone needs the largest model. There are variations or R1 with fewer parameters that can easily run on consumer hardware. With 80% size reduction you could run 70B on 8-bit on an RTX 3090. Other than that, if you really need the big one you can get six 3090s and you're good to go. It's not cheap, but you're running a ChatGPT equivalent model from your basement. A year ago this was a wetdream for most enthusias…
Or if you want a large model but don’t need high performance, get a Mac with 128GB UMA.
Has it been tried on 128GB M4 MacBook Pro? I'm gonna try it, but I guess it will be too slow to be usable. I love the original DeepSeek model, but the distilled versions are too dumb usually. I'm excited to try my own queries on it.
I'm downloading it now and will report back. (I've been using the 32B and while it could always be better, I'm not unhappy with it)