Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run DeepSeek R1 Dynamic 1.58-bit

unsloth.ai

Post reply on HN
Print view View original

261–270 of 346 posts

Re: Run DeepSeek R1 Dynamic 1.58-bit

#261

Post by cubefox »

chris_pie wrote: post #211
stackedinserter wrote: post #121

Earlier quoted context omitted.

It sounds like if they owe you the training architecture and training data set.

It absolutely doesn't. It sounds like further diluting the term "open-source" isn't great.

I assume when people say "open source model" they mean "open weights model". The "open source" term doesn't really make sense here, since machine learning models are not compilations of source code. (Though DeepSeek has published several papers with details on their training process. It's more than just open weights.)

Re: Run DeepSeek R1 Dynamic 1.58-bit

#262

Post by rahimnathwani »

Pxtl wrote: post #248

Is there any good quick summary of what's special about DeepSeek? I know it's OSS and incredibly efficient, but news laymen are saying it's trained purely on AI info instead of using a corpus of tagged data... which, I assume, means it's somehow extracting weights or metadata or something from other AIs. Is that it?

  Is there any good quick summary of what's special about DeepSeek?
Yes, section 2.3 of the Deepseek R1 paper summarizes the training part you're asking about, in less than a page.

https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSee...

Re: Run DeepSeek R1 Dynamic 1.58-bit

#263

Post by rahimnathwani »

xiphias2 wrote: post #236

Has it been tried on 128GB M4 MacBook Pro? I'm gonna try it, but I guess it will be too slow to be usable. I love the original DeepSeek model, but the distilled versions are too dumb usually. I'm excited to try my own queries on it.

  I love the original DeepSeek model, but the distilled versions are too dumb usually.
Apart from being dumber, they also don't know as much as R1. I can see how fine-tuning can improve reasoning capability (by showing examples of good CoT) but there's no reason that would improve the knowledge of facts (relative to the Qwen or Llama model on which the finetuning was based).

Re: Run DeepSeek R1 Dynamic 1.58-bit

#264

Post by jadbox »

sourcecodeplz wrote: post #242
jmb99 wrote: post #210

Earlier quoted context omitted.

It’s more expensive, but Zen4 Threadripper Pro is probably the way to go on that front. 8 memory channels, with DIMMs available up to DDR5-7200 for 8x32GB (256GB), or DDR5-6800 for 8x48GB (384GB). It’ll set you back ~$3k for the RAM and ~$6k for a CPU with 8 CCDs (the 7985WX, at least), and then ~$1k for motherboard and however much you want to spend on NVME. Basically ~$10k for a 384GB DDR5 system with ~435GB/s actu…

At these prices, I would just get 2xDigits for $6k and have 256gb.

I have a feeling that Digits will probably get sold out and will pricing will get hiked WAY up.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#265

Post by Der_Einzige »

smcleod wrote: post #69
danielhanchen wrote: post #49

Earlier quoted context omitted.

Oh the repetition issue is only on the non dynamic quants :) If you do dynamic quantization and use the 1.58bit dynamic quantized model the repetition issue fully disappears! Min_p = 0.05 was a way I found to counteract the 1.58bit model generating singular incorrect tokens which happen around 1 token per 8000!

min_p is great, do you apply a small amount of temperate as well?

Btw, min_p (the paper about the sampler) got accepted to ICLR! As 4th author it warms my heart to so it used so much in the wild.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#266

Post by dimgl »

sailingparrot wrote: post #131
yobid20 wrote: post #91

Earlier quoted context omitted.

I canceled my OpenAI subscription last night, as did many many others. There were some threads in reddit with everyone chiming in they all just canceled too. imo OpenAI is done, and will go through massive cuts and probably acquired by the end of the year for a very tiny fraction of its current value.

You want to bet? The panic around deepseek is getting completely disconnected from reality. Don’t get me wrong what DS did is great, but anyone thinking this reshape the fundamental trend of scaling laws and make compute irrelevant is dead wrong. I’m sure OpenAI doesn’t really enjoy the PR right now, but guess what OpenAI/Google/Meta/Anthropic can do if you give them a recipe for 11x more efficient training ? They ca…

Deepseek is not the only reason. I cancelled my OpenAI subscription because I've replaced it wholesale with Anthropic.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#267

Post by DogRunner »

>For optimal performance, we recommend the sum of VRAM + RAM to be at least 80GB+.

Oh nice! So I can try it in my local "low power/low cost" server at home.

My homesystem does run in a ryzen 5500 + 64gb RAM + 7x RTX 3060 12gb

So 64gb RAM plus 84gb VRAM

I dont want to brag around, but point to solutions for us tinkerers with a small budget and high energy costs.

such system can be build for around 1600 euro. The power consumption is around 520 watt.

I started with a AM4 Board (b450 Chipset) and one used RTX 3060 12gb which cost around 200 Euro used if you are patient.

There every additional GPU is connected with the pcie riser/extender to give the cards enough space.

After a while I had replaces the pcie cards with a single pcie x4 to 6x PCIe x1 extender.

It runs pretty nice. Awesome to learn and gain experience

Re: Run DeepSeek R1 Dynamic 1.58-bit

#268

Post by nicce »

yobid20 wrote: post #91
apples_oranges wrote: post #16

Random observation 1: I was running DeepSeek yesterday on my Linux with a RTX 4090 and I noticed that the models should fit into VRAM, which is 24GB. Or they are simply slow. So the Apple shared memory architecture has an advantage here. A 192GB Mx Ultra can load and process large models efficiently. Random observation 2: It's time to cancel the OpenAI subscription.

I canceled my OpenAI subscription last night, as did many many others. There were some threads in reddit with everyone chiming in they all just canceled too. imo OpenAI is done, and will go through massive cuts and probably acquired by the end of the year for a very tiny fraction of its current value.

1. You can get all the models by buying Kagi subscription (excluding o1). Includes DeepSeek models. You can also feed the assistant with search data that you can filter.

2. If you have GitHub Copilot, you get o1 chat also there.

I haven't seen much value with OpenAI subscription for ages.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#269

Post by FranchuFranchu »

michaelt wrote: post #123
orf wrote: post #73

Earlier quoted context omitted.

Laptops get stolen on a train? An enclosed, single-direction space that only occasionally allows you to exit between infrequent, long-distance stops? A thing that contains ticket inspectors and a literal guard? How many laptops have you personally seen be stolen on a train?

You don't hear much about laptop thefts these days because phones are more valuable, more numerous, and much easier to steal. Obviously, nobody steals things while the train is in motion. They wait until the train is about to leave the station, snatch a phone or handbag and jump out just as the door is closing. The train leaves, the thief blends in with other passenger leaving the station, and by the time news of the…

> Obviously, nobody steals things while the train is in motion.

Something interesting: I live near a train line where the doors are not automatic (they have to be opened manually on each stop), and there have been incidents where people get pickpocketed while the train is still in motion, and the thief jumps out right before the station, when the train has slowed down significantly but is still in motion. Many people have been hurt doing this.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#270

Post by nicce »

wqaatwt wrote: post #148
kebaman wrote: post #133

Earlier quoted context omitted.

Doesn't Microsoft own 49% of OpenAI? They'll end up with it all as a division of Microsoft.

I think they “own” 49% of OpenAI’s net income until a certain very high amount. Not a share of the actual company.

They "own" even 75% of profits until Microsoft has recouped its $13 billion investment. 49% comes after that.
Post reply on HN