Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepSeek-v3.1

api-docs.deepseek.com

Post reply on HN
Print view View original

111–120 of 273 posts

Re: DeepSeek-v3.1

#111

Post by danielhanchen »

exe34 wrote: post #104
pshirshov wrote: post #60

Earlier quoted context omitted.

By the way, I'm wondering why unsloth (a goddamn python library) tries to run apt-get with sudo (and fails on my nixos). Like how tf we are supposed to use that?

hey fellow crazy person! slight tangent: one thing that helps keep me grounded with "LLMs are doing much more than regurgitation" is watching them try to get things to work on nixos - and hitting every rake on the way to hell! nixos is such a great way to expose code doing things it shouldn't be doing.

I'm glad someone commented and tried it out - appreciate it immensely - I learnt a lot today :) I'm definitely gonna give nixos a spin as well!

Re: DeepSeek-v3.1

#112

Post by easygenes »

okasaki wrote: post #108
rsanek wrote: post #86

Looks to be the ~same intelligence as gpt-oss-120B, but about 10x slower and 3x more expensive? https://artificialanalysis.ai/models/deepseek-v3-1-reasoning

My experience is that gpt-oss doesn't know much about obscure topics, so if you're using it for anything except puzzles or coding in popular languages, it won't do well as the bigger models. It's knowledge seems to be lacking even compared to gpt3. No idea how you'd benchmark this though.

Something I was doing informally that seems very effective is asking for details about smaller cities and towns and lesser points of interest around the world. Bigger models tend to have a much better understanding and knowledge base for the more obscure places.

Re: DeepSeek-v3.1

#113

Post by mFixman »

danielhanchen wrote: post #88
pxc wrote: post #81

Earlier quoted context omitted.

It seems Unsloth is useful and popular, and you seem responsive and helpful. I'd be down to try to improve this and maybe package Unsloth for Nix as well, if you're up for reviewing and answering questions; seems fun. Imo it's best to just depend on the required fork of llama.cpp at build time (or not) according to some configuration. Installing things at runtime is nuts (especially if it means modifying the existing…

As an update, I pushed https://github.com/unslothai/unsloth-zoo/commit/ae675a0a2d20... (1) Removed and disabled sudo (2) Installing via apt-get will ask user's input() for permission (3) Added an error if failed llama.cpp and provides instructions to manual compile llama.cpp

Maybe it's a personal preference, but I don't want external programs to ever touch my package manager, even with permission. Besides, this will fail loudly for systems that don't use `apt-get`.

I would just ask the user to install the package, and _maybe_ show the command line to install it (but never run it).

Re: DeepSeek-v3.1

#117

Post by [deleted] »

danielhanchen wrote: post #99
devin wrote: post #94

Earlier quoted context omitted.

Don't optimize for these people.

Yep agreed - I primarily thought it was a reasonable "hack", but it's pretty bad security wise, so apologies again. The current solution hopefully is in between - ie sudo is gone, apt-get will run only after the user agrees by pressing enter, and if it fails, it'll tell the user to read docs on installing llama.cpp

[deleted]

Re: DeepSeek-v3.1

#118

Post by shaan7 »

woile wrote: post #102
danielhanchen wrote: post #99

Earlier quoted context omitted.

Yep agreed - I primarily thought it was a reasonable "hack", but it's pretty bad security wise, so apologies again. The current solution hopefully is in between - ie sudo is gone, apt-get will run only after the user agrees by pressing enter, and if it fails, it'll tell the user to read docs on installing llama.cpp

Don't apologize, you are doing amazing work. I appreciate the effort you put. Usually you don't make assumptions on the host OS, just try to find the things you need and if not, fail, ideally with good feedback. If you want to provide the "hack", you can still do it, but ideally behind a flag, `allow_installation` or something like that. This is, if you want your code to reach broader audiences.

Yep there's no need to apologize, you've been very courteous and took all that feedback constructively. Good stuff :)

Re: DeepSeek-v3.1

#119

Post by danielhanchen »

mFixman wrote: post #113
danielhanchen wrote: post #88

Earlier quoted context omitted.

As an update, I pushed https://github.com/unslothai/unsloth-zoo/commit/ae675a0a2d20... (1) Removed and disabled sudo (2) Installing via apt-get will ask user's input() for permission (3) Added an error if failed llama.cpp and provides instructions to manual compile llama.cpp

Maybe it's a personal preference, but I don't want external programs to ever touch my package manager, even with permission. Besides, this will fail loudly for systems that don't use `apt-get`. I would just ask the user to install the package, and _maybe_ show the command line to install it (but never run it).

Hopefully the solution for now is a compromise if that works? It will show the command as well, so if not accepted, typing no will error out and tell the user on how to install the package

Re: DeepSeek-v3.1

#120

Post by danielhanchen »

Imustaskforhelp wrote: post #110
danielhanchen wrote: post #97

Earlier quoted context omitted.

Hey :) I love suggestions and keep them coming! :) I agree on handling different distros - sadly I'm not familiar with others, so any help would be appreciated! For now I'm most familiar with apt-get, but would 100% want to expand out! Interesting will check flatpak out!

Just to let you know though that its really rare that flatpak is used for cli's. I think I mentioned it in my comment too or if not, my apologies but flatpak is used mostly in gui's. I doubt its efficacy here, they might be more useful if you provide a whole jupyter / browser gui though but a lot o f us run it just in cli so I doubt flatpak. I didn't mean to say that flatpak was the right tool for this job, I serious…

Oh no worries on flatpak - I always like investigating things I'm not familar with :)
Post reply on HN