Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

yomaq.github.io

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#31

Post by neuroelectron »

csdreamer7 wrote: post #29
throwawayqqq11 wrote: post #24

Earlier quoted context omitted.

Not if you can migrate to linux on a whim.

^This. People act so helpless to big tech, and especially Windows. Linux is still more effort for normies, but it should not be an issue for most people on this sub to want learn it-you are doing things already on Windows that would not be much different than Linux. AI w/ Nividia alone justifies Linux because Windows has performance issues. Then you have the freedom of leaving Windows if it continues to get hostile.

Brave to assume linux is more secure than Windows. I didn't read the rest of your comment.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#32

Post by rad_gruchalski »

csdreamer7 wrote: post #29
throwawayqqq11 wrote: post #24

Earlier quoted context omitted.

Not if you can migrate to linux on a whim.

^This. People act so helpless to big tech, and especially Windows. Linux is still more effort for normies, but it should not be an issue for most people on this sub to want learn it-you are doing things already on Windows that would not be much different than Linux. AI w/ Nividia alone justifies Linux because Windows has performance issues. Then you have the freedom of leaving Windows if it continues to get hostile.

Unless you're working for a company where you have no choice. Then you have no freedom to leave Windows. You have, obviously, the freedom to leave your job, but let's be adult.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#34

Post by calaphos »

emsign wrote: post #22

Running models at home seems like a waste of money while at the same time they are currently heavily subsidized in the cloud by dumb money.

There's also big efficiency increases when batching multiple requests, making clouds inherently more cost effective for normal use cases.

Way better utilization of expensive hardware as well ofc.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#35

Post by elwebmaster »

I would dare guess the author just doesn’t know there is a perfectly functional Windows native Ollama release. I was doing the same thing until I realized that it makes no sense because I can just install ollama on Windows and then connect to it from within WSL.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#36

Post by hu3 »

borisk wrote: post #25
emsign wrote: post #22

Running models at home seems like a waste of money while at the same time they are currently heavily subsidized in the cloud by dumb money.

How is it a waste of money if a lot of people, including the author of the article, already have an nVidia GPU in their PC? Running locally has a lot of advantages - privacy, getting to learn how to run LLMs, not having to deal with quotas, logins, outages.

Home electricity bill of a decent GPU alone is usually more expensive than renting GPUs on demand.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#37

Post by razemio »

nanozuki wrote: post #18
razemio wrote: post #7

Just to awnser the question why not using windows. Here are some ideas why the author might have used nix instead: - reproducible (with minor adjustments even on non-WSL systems) - if you are used to nix, there is not much which beats it in terms of stability, maintainability, upgradability, and fun (?) - additional services are typically easier to set up like tailscale-acl, used by the author, which uses pulumi unde…

If the author like NixOS, they can use NixOS in native, and then they can get native Ollama again. No need to run nix on WSL

They can not game on it (yet). That was the point of this post. Having a 2 in 1 device. For gaming and on demand ai.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#38

Post by razemio »

baq wrote: post #13
razemio wrote: post #7

Just to awnser the question why not using windows. Here are some ideas why the author might have used nix instead: - reproducible (with minor adjustments even on non-WSL systems) - if you are used to nix, there is not much which beats it in terms of stability, maintainability, upgradability, and fun (?) - additional services are typically easier to set up like tailscale-acl, used by the author, which uses pulumi unde…

IME storage speed is only an issue with if you’re using the shared folder capability. It’s as fast as any vm if you stick to the wsl block device.

Correct. I had the issue that I developed a windows native Software but wanted to use my linux Tools.

Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC

#39

Post by IHLayman »

I love the idea of this flake to run Ollama even on Windows, but just pointing people to your _everything_ flake is going to confuse people and make it look harder than it is to run Ollama on Nix.

If you are using a system-controlling Nix (nix-darwin, NixOS…), it’s as easy as `hardware.services.ollama.enable=true` with maybe adding `.acceleration=“cuda”` to force GPU usage or `host=“0.0.0.0”` to allow connections to Ollama that are not local to your system. In a home-manager situation it is even easier: just include `pkgs.ollama` in your `home.packages`, with an `.override{}` for the same options above. That should be it, really.

I will say that if you have a more complex NixOS setup that patches the kernel or can’t lean on cachix for some reason that using the ollama package takes a long time to compile. My setup at home runs on a 3950X Threadripper and when Ollama compiles it uses all the cores at 99% for about 16 minutes.

Post reply on HN