Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

yomaq.github.io

Post reply on HN
Print view View original

21–30 of 51 posts

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#21

Post by neuroelectron »

benterix wrote: post #19
neuroelectron wrote: post #14

Earlier quoted context omitted.

Reproducability and security. You're getting the managed security of a Windows install plus your isolating it from Windows telementry.

> your isolating it from Windows telementry 1. Maybe, 2. For now

Yes that's true but that's how it always is.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#25

Post by borisk »

emsign wrote: post #22

Running models at home seems like a waste of money while at the same time they are currently heavily subsidized in the cloud by dumb money.

How is it a waste of money if a lot of people, including the author of the article, already have an nVidia GPU in their PC?

Running locally has a lot of advantages - privacy, getting to learn how to run LLMs, not having to deal with quotas, logins, outages.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#28

Post by itissid »

I've been running an ollama and deepseek in a container in TrueNAS k8s for several months. It's hooked up to my Continue extension in VSCode. I also mix it with cloud hosted "dumb" ones for other tasks like code completion. Ollama deepseek is reserved for heavier chat and code tasks.

It's fast as hell. Though you will need at least two GPUs to divide between ollama and if need something else(display/game/proxmox) to use it.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#29

Post by csdreamer7 »

throwawayqqq11 wrote: post #24
neuroelectron wrote: post #21

Earlier quoted context omitted.

Yes that's true but that's how it always is.

Not if you can migrate to linux on a whim.

^This. People act so helpless to big tech, and especially Windows. Linux is still more effort for normies, but it should not be an issue for most people on this sub to want learn it-you are doing things already on Windows that would not be much different than Linux. AI w/ Nividia alone justifies Linux because Windows has performance issues. Then you have the freedom of leaving Windows if it continues to get hostile.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#30

Post by chpatrick »

emsign wrote: post #22

Running models at home seems like a waste of money while at the same time they are currently heavily subsidized in the cloud by dumb money.

I use ChatGPT for most practical stuff but I really enjoy running local models. I find it really interesting and I think it's important for people to know how to run these without being beholden to big tech. If you have a used 3090 you can already run some really strong models. There are some really interesting local models as well, like the abliterated ones.
Post reply on HN