Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Run DeepSeek R1 Dynamic 1.58-bit

unsloth.ai

Post reply on HN
Print view View original

221–230 of 346 posts

Re: Run DeepSeek R1 Dynamic 1.58-bit

#221

Post by afavour »

TeMPOraL wrote: post #120
afavour wrote: post #105

Earlier quoted context omitted.

Pickpocketing is a very different proposition. They relying on a lack of awareness, taking your wallet and being long gone before you’ve even noticed. If someone steals your laptop from in front of you without you even noticing I’d suggest that one is on you. FWIW I’ve used my laptop on the train plenty, I’ve never had anything stolen nor felt in any danger of it.

But would you consider leaving it unattended on your seat and going for lunch to the restaurant car, or for an extended toilet break?

...why would I ever do that? You leave something worth several thousand dollars anywhere in public you're risking losing it. What are we even debating here?

Re: Run DeepSeek R1 Dynamic 1.58-bit

#222

Post by whimsicalism »

mcv wrote: post #220
whimsicalism wrote: post #199

Earlier quoted context omitted.

do you only take advice/learn from all-knowing people?

Do you know any? But here's my advice: drop the fallacious arguments and try something more honest.

my argument isn’t fallacious - it is logical: we can learn/use evidence from something without presuming it is all knowing. you are putting words in others mouths that they did not say

Re: Run DeepSeek R1 Dynamic 1.58-bit

#223

Post by immibis »

dagaci wrote: post #155
therealpygon wrote: post #136

Earlier quoted context omitted.

I honestly don’t know why people aren’t more upset by this and still get on their knees for Nvidia. They made the decision specifically to cripple consumer card memory because they didn’t like data centers were using them instead of buying their overpriced enterprise cards that were less performant. They removed NVLink because people were getting better performance out of their two $400 cards than the $1,500 cards Nv…

Because sensible people just use the cloud at this point, you can probably get several years of training for $6000

It buys you approximately two days (with reservation discount) of a single p5.48xlarge instance, which has 2TB of RAM, and 640GB of VRAM in 8x H100 cards. In fact that is the pricing example they use: https://aws.amazon.com/ec2/capacityblocks/pricing/

Re: Run DeepSeek R1 Dynamic 1.58-bit

#224

Post by vladms »

crazygringo wrote: post #184
vladms wrote: post #129

Earlier quoted context omitted.

Different regions of the world would see different degrees of responsibilities regarding theft. I would consider absurd to leave unattended in a public space something valuable, considering the effort required to avoid that (that is: taking it with you). So, yes, theft on trains for people that think they are 100% safe are a thing, but applying the same idea (to assume something is 100% safe and not be cautious) I wo…

You're going to take your laptop with you into the toilet on the train...? I don't think I've ever seen a human being do that before on a train. Not to go to the toilet, nor to grab a coffee in another car. You can't be paranoid about everything. My friend in France had put his laptop back into his bag where it wasn't visible and assumed that was good enough, but someone must have seen him do it and just took the who…

Yes, if I go to the toilet I take my backpack/small bag with me, because usually I have valuable stuff in them and are easy to carry. This does not apply to a large bag (in which I don't put valuable stuff).

The standard is mine and I follow it. The same way I find absurd not to do it, you find it unreasonable to do it.

I find the expectation that things are not stolen (if unsupervised in public places) strange considering the huge amount of inequalities in wealth around even in civilized countries. I do not agree with the idea of stealing, thiefs should be punished, but expecting everybody "to behave" given the situation seems unrealistic to me.

That does not mean that I think that things are stolen 100% of the time. I have a friend that forgot a laptop on a bus (Netherlands) and the driver found it at the end of the line and gave it to lost objects so my friend got it back.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#225

Post by vladms »

Thorrez wrote: post #147
vladms wrote: post #129

Earlier quoted context omitted.

Different regions of the world would see different degrees of responsibilities regarding theft. I would consider absurd to leave unattended in a public space something valuable, considering the effort required to avoid that (that is: taking it with you). So, yes, theft on trains for people that think they are 100% safe are a thing, but applying the same idea (to assume something is 100% safe and not be cautious) I wo…

My coworker was having coffee and using his work laptop at an outdoor coffeeshop in Mountain View, CA. Someone on a bike rode by and attempted grab his phone and bike off with it. The attempted thief didn't succeed in taking the phone, but did knock the laptop onto the ground, damaging it.

The discussion was about leaving unattended valuable objects in public places. Sure, a theft can happen even if attended, or using violence, but I personally avoid increasing the chance of having something stolen by leaving it unattended.

If I would make a statistics of primary cause of remaining without a laptop among people I know, the biggest danger is liquids in glasses (that ends up on the laptops) ...

Re: Run DeepSeek R1 Dynamic 1.58-bit

#226

Post by Majromax »

tarruda wrote: post #56
huijzer wrote: post #40

Earlier quoted context omitted.

> Is the claim that it only took $5M to train generally accepted? Based on Nvidia being down 18% yesterday I would say the claim is generally accepted.

It is still unconfirmed since no one outside of deepseek reproduced it. If confirmed, Nvidia could go down even more

I'm not sure I see the bear argument for NVidia here. Huge AI models certainly drive NVidia sales, but huge AI models are also widely thought to be untrainable and nearly un-runnable save for large datacenters.

To me, this is ripe for an application of the Jevons paradox. If architectural improvements make similar models cheaper, I would expect to see more of them trained and deployed, not fewer, ultimately increasing the market for GPU-like hardware.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#227

Post by ddingus »

bnchrch wrote: post #200
F7F7F7 wrote: post #197

Earlier quoted context omitted.

People would only be 'throwing their hands up' because commercial LLMs have set unreasonable expectations for folks. Anyone who has a/the need for or understands the value of a local LLM would be OK with this kind of output.

Everyone has the need for on device LLM, if the response rate was fast!

I have MLCCHAT on my old Note 9 phone. It is actually still a great phone, but has 5GB RAM. Running an on device model is the first and only use case the RAM actually matters.

And it has a headphone jack, OK? I just hate Bluetooth earbuds. And yeah, it isna problem, but I digress.

When I run a 2.5B model, I get respectable output. Takes a minute or two to process the context, then output begins at somewhere on the order of 4 to 10 tokens per sec.

So, I just make a query and give it a few and I have my response.

Here is how I see it:

That little model, which is Gemma 2.2b sorry, knows a lot of stuff. It has knowledge I don't and it gives it to me in a reasonable, though predictable way. Answers are always of a certain teacher reminding student how it all goes way.

I don't care. Better is nice, but if I were stuck somewhere with no network, being able to query that model is amazing!

First aid, how to make fires, materials and uses. Fixing stuff, theories of operation, what things mean and more are in that thing ready for me to take advantage of.

I consider what I have fast. And it will get one or two orders faster over the next few years too.

I did it on a lark (ask the model what that means) and was surprised to see I gained a nice tool.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#228

Post by collingreen »

Thorrez wrote: post #147
vladms wrote: post #129

Earlier quoted context omitted.

Different regions of the world would see different degrees of responsibilities regarding theft. I would consider absurd to leave unattended in a public space something valuable, considering the effort required to avoid that (that is: taking it with you). So, yes, theft on trains for people that think they are 100% safe are a thing, but applying the same idea (to assume something is 100% safe and not be cautious) I wo…

My coworker was having coffee and using his work laptop at an outdoor coffeeshop in Mountain View, CA. Someone on a bike rode by and attempted grab his phone and bike off with it. The attempted thief didn't succeed in taking the phone, but did knock the laptop onto the ground, damaging it.

A random person able to dart in and then make a getaway is not what "working on a train" is like and that was the original comment's point.

Re: Run DeepSeek R1 Dynamic 1.58-bit

#229

Post by ddingus »

bnchrch wrote: post #200
F7F7F7 wrote: post #197

Earlier quoted context omitted.

People would only be 'throwing their hands up' because commercial LLMs have set unreasonable expectations for folks. Anyone who has a/the need for or understands the value of a local LLM would be OK with this kind of output.

Everyone has the need for on device LLM, if the response rate was fast!

I fired the model up at 8:39, ready to chat just a bit before 8:40, during which I input the following prompt:

Please define "lark" for me, as in "I did it on a lark."

Output began a bit before 8:41, and was complete before 8:43:

---Begin Model Output

"On a lark" means without much planning or reason, impulsively or unexpectedly.

Here's how it works:

"I did it on a lark." This means the person acted without a specific plan or reason. "A lark" refers to the lark bird, a small songbird known for its playful nature.

So, "on a lark" suggests something spontaneous, casual, and often with a touch of playful whimsy.

Here are some other ways to understand it:

Without much thought: The person was likely to do it quickly and without much consideration. Based on a whim: It's something they decided to do without a solid plan.

Let me know if you'd like more examples

---End Model Output

To me, that is pretty damn spiffy!

Re: Run DeepSeek R1 Dynamic 1.58-bit

#230

Post by miroljub »

sailingparrot wrote: post #131
yobid20 wrote: post #91

Earlier quoted context omitted.

I canceled my OpenAI subscription last night, as did many many others. There were some threads in reddit with everyone chiming in they all just canceled too. imo OpenAI is done, and will go through massive cuts and probably acquired by the end of the year for a very tiny fraction of its current value.

You want to bet? The panic around deepseek is getting completely disconnected from reality. Don’t get me wrong what DS did is great, but anyone thinking this reshape the fundamental trend of scaling laws and make compute irrelevant is dead wrong. I’m sure OpenAI doesn’t really enjoy the PR right now, but guess what OpenAI/Google/Meta/Anthropic can do if you give them a recipe for 11x more efficient training ? They ca…

> You want to bet?

Why would anyone bet? They can just short the OpenAI / MS stocks, and see in a few months if they were right or not.

Post reply on HN