Gaming PC can run Linux to begin with.
Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
11–20 of 51 posts
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#12This is great work, it solves an exact problem I too am having. Now I just need to upgrade my 12 year old GPUs to something that can run an LLM.
In my case, I would have to upgrade my >12 years old PC. :D
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#13Just to awnser the question why not using windows. Here are some ideas why the author might have used nix instead: - reproducible (with minor adjustments even on non-WSL systems) - if you are used to nix, there is not much which beats it in terms of stability, maintainability, upgradability, and fun (?) - additional services are typically easier to set up like tailscale-acl, used by the author, which uses pulumi unde…
IME storage speed is only an issue with if you’re using the shared folder capability. It’s as fast as any vm if you stick to the wsl block device.
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#14I agree with the other commenters, this post does not explain why you would not just run Ollama or Koboldcpp on Windows. What exactly makes running Ollama within virtualized NixOS in WSL in some way better than running natively? If it's just the novelty aspect of it or some idealogical reason, that's fine, but it should be explained in the blog post before someone thinks this is a sane and logical way to run Ollama o…
Reproducability and security. You're getting the managed security of a Windows install plus your isolating it from Windows telementry.
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#15Nvidia never fixed their sysmem fallback policy for wsl2 though, running on wsl2 rather than native Windows just spell so much performance problems when VRAM overflows
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#16Gaming PC can run Linux to begin with.
That's what mine does as it's a gaming PC secondarily.... that being said I am running in to some limitations i.e. certain Sim Racing titles and VR Support (though both ALVR and WiVRN are excellent)
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#17>gaming PC
>LLM
stinky
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#18Just to awnser the question why not using windows. Here are some ideas why the author might have used nix instead: - reproducible (with minor adjustments even on non-WSL systems) - if you are used to nix, there is not much which beats it in terms of stability, maintainability, upgradability, and fun (?) - additional services are typically easier to set up like tailscale-acl, used by the author, which uses pulumi unde…
If the author like NixOS, they can use NixOS in native, and then they can get native Ollama again. No need to run nix on WSL
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#19I agree with the other commenters, this post does not explain why you would not just run Ollama or Koboldcpp on Windows. What exactly makes running Ollama within virtualized NixOS in WSL in some way better than running natively? If it's just the novelty aspect of it or some idealogical reason, that's fine, but it should be explained in the blog post before someone thinks this is a sane and logical way to run Ollama o…
Reproducability and security. You're getting the managed security of a Windows install plus your isolating it from Windows telementry.
> your isolating it from Windows telementry
1. Maybe, 2. For now
Re: Nvidia on NixOS WSL – Ollama up 24/7 on your gaming PC
#20Just to awnser the question why not using windows. Here are some ideas why the author might have used nix instead: - reproducible (with minor adjustments even on non-WSL systems) - if you are used to nix, there is not much which beats it in terms of stability, maintainability, upgradability, and fun (?) - additional services are typically easier to set up like tailscale-acl, used by the author, which uses pulumi unde…
If the author like NixOS, they can use NixOS in native, and then they can get native Ollama again. No need to run nix on WSL
The clue seems to be "on your gaming PC". Many linux users still run Windows on their gaming machines due to compatibility. This would also probable be their device which has the most powerful GPU.