81–90 of 162 posts
This continues the pattern of all other announcements of running 'Deepseek R1' on raspberry pi - that they are running llama (or qwen), modified by deepseek's distillation technique.
Earlier quoted context omitted.
And DeepSeek itself is (allegedly) a distillation of OpenAI models.
Never heard that claim before, only that a certain subset of re-enforced learning may have used ChatGPT to grade responses. Is there more detail about it being allegedly a distilled OpenAI model?
Earlier quoted context omitted.
And DeepSeek itself is (allegedly) a distillation of OpenAI models.
Never heard that claim before, only that a certain subset of re-enforced learning may have used ChatGPT to grade responses. Is there more detail about it being allegedly a distilled OpenAI model?
Earlier quoted context omitted.
You can buy it but you can't run it, unless you're fairly wealthy. In my country (italy) a basic colocation service is like 80 euros/month + vat, and that only includes 100Wh of power and a 100mbps connection. +100wh/month upgrades are like +100 euros. I looked up the kind of servers and cpus you're talking about and the cpu alone can pull something like 180W/h, without accounting for fans, disks and other stuff (stu…
Interesting, I run a small cluster of 4 mini pcs (22 cores total). I think it should be comparable to the aforementioned EPYC. Power load is a rounding error compared to appliances like electric iron at 1700W, etc. The impact on electrical bill is minimal as well. Idle power draw is about 5W per server, which translates to ~80 cents a month. Frankly my monitor uses more power on average than the rest of the homelab.
Really there needs to be a product based off of LLMs similar to Alexa or Google home where instead of connecting to the cloud it’s a locally run LLM. I don’t know why one doesn’t exist yet or why no one is working on this
You mean like Ollama + llamacpp ?
This continues the pattern of all other announcements of running 'Deepseek R1' on raspberry pi - that they are running llama (or qwen), modified by deepseek's distillation technique.
Earlier quoted context omitted.
Can you explain to a non ML software engineer what these distillation methods mean? What does it mean to have R1 train a Llama model? What is special about DeepSeek’s distillation methods? Thanks!
Its llama/quen with some additional training to add reasoning. In a similar way deep seeks v3 was trained into r1. It also looks to me like there was some Chinese propaganda trained into llama/quen too, but that’s just my observation.
This continues the pattern of all other announcements of running 'Deepseek R1' on raspberry pi - that they are running llama (or qwen), modified by deepseek's distillation technique.
Yes this is just a fine-tuned LLaMa with DeepSeek-like "chain of thought" generation. A properly 'distilled' model is supposed to be trained from scratch to completely mimick the larger model it's being derived from - which is not what's going on here.