Earlier quoted context omitted.
You need to turn off the default conda environment. Try: conda config --set auto_activate_base false
Thanks !!!
:)
Windows AI Studio Preview
61–69 of 69 posts
Re: Windows AI Studio Preview
#62Earlier quoted context omitted.
> There's an 80GB GPU out there that costs about six times as much as an equivalent Mac with 96 GB of shared RAM. Those are not equivalent in speed. macs RAM are much slower than these GPUs.
There is no Apple-to-apples comparison between a Macbook and a GPU. However, the problem with running most models is the lack of simultaneous RAM. You can swap memory back and forth between RAM and VRAM (with a huge performance penalty) of course, but that's not exactly usable or comparable to what Apple's VRAM sharing setup allows. Nvidia doesn't sell a nice-but-not-amazing GPU equivalent to Apple's processing power…
That sounds like an overly specific target audience:
- you need more than 16 or 32 GB of ram.
- but less than 100.
- speed is important.
- but losing a factor 4 compared to a GPU is fine.
- money is very limited.
- but paying 6000 for a mac studio is cheap.
- this is very important professional work.
- but I don't need servers, ECC ram, raid, another OS...
Don't get me wrong, I also think Nvidia is heavily price gouging, but there are definitely trade offs and it is not clear at all why 80GB is your magical number and not, say, 108 or 28.
Re: Windows AI Studio Preview
#63Earlier quoted context omitted.
Yep, didn't read that. In the meantime, kudos to Canonical (and other parties) who helped Microsoft with WSL. Now they can also use Linux to produce Windows-only software! :D
The effort always stops when the commercial goal is achieved.
I don't know, once upon a time I would have agreed with you, but recently I'm optimistic that MS is genuinely trying to break out of that behavior. Time will tell.
Re: Windows AI Studio Preview
#64Earlier quoted context omitted.
that probably explain why nvidia only. Amd don't really have rocm for linux on latest consumer cards.
The Nvidia container toolkit works pretty much out of the box on WSL these days as well. Funny that some cuda stuff works better through Windows virtualizing Linux than Windows natively, but if we're being honest even as a native Linux user, WSL probably provides a better user experience (vs having to use Nvidia drivers on Linux anyways)
Hasn't been my experience. Windows keeps a few hundred mb of vram tied up for the gui, Linux only holds 14mb per GPU
Re: Windows AI Studio Preview
#65Earlier quoted context omitted.
Cuda would be pointless for Apple, because cuda is for Nvidia only, no? As for the price - when you get to 32/64/96GB levels of RAM it’s the cheapest setup on the market that can get you this much VRAM. At least that’s what it was half a year ago when I checked the last time
> Cuda would be pointless for Apple, because cuda is for Nvidia only, no? That is the point. Cuda is widely used in particular for training as opposed to tuning and is just flat out not available. So you buy your nice $6000 machine and it just does not work for its intended purpose. > it’s the cheapest setup on the market that can get you this much VRAM. shared VRAM is not everything.
Shared vram is everything of you're running local inference.
My 3 x 3090s can't fit the 180b falcon even at 2 bits into vram. But people with a 128gb studio can, and it's cheaper
Re: Windows AI Studio Preview
#66Earlier quoted context omitted.
Is the container toolkit even necessary anymore in WSL2? I did a fresh install of windows 10 LTSC a few weeks ago and installed WSL2 and docker with WSL2 integration and I was able to use my nvidia rtx card (checked through nvidia-smi) without any issues.
It registered the card, as in you can query the device id using nvidia-smi. But CUDA/cuDNN requires further work.
Interesting. I'll have to check to be sure, but I think maybe something is happening automagically if you have reasonably up to date nvidia drivers on the host OS, because I was able to run the EmotiVoice TTS docker (which requires nvidia gpu) from WSL2.
Re: Windows AI Studio Preview
#67Earlier quoted context omitted.
But can you get this to run in Linux? From the article I can see it's a VSCode extension or something like that but the name kind of implies it's Windows only.
The link takes me to marketplace.visualstudio.com which makes it seem like a Visual Studio extension, not a VSCode extension? Not sure, it's as confusing as the live/hotmail/outlook/authentication parts of Microsoft tends to be. Besides, it says the following: > Windows AI Studio will run only on NVIDIA GPUs for the preview, so please make sure to check your device spec. WSL Ubuntu distro 18.4 or greater should be in…
> So seems it'll only run in Linux, as it's a requirement to have Ubuntu WSL running beforehand.
Yes, it'll only run in Linux, but only when that Linux is also running inside Windows.
Re: Windows AI Studio Preview
#68Windows AI Studio is in real a Linux AI Studio as it needs WSL to run. Little funny
Obligatory "is this the year of Linux in desktop" joke
It is kind of ironic that the Year of Linux on the Desktop is all based on having Linux VMs across all major mainstream desktop OSes, even on ChromeOS, Crostini runs inside something similar to WSL.
Re: Windows AI Studio Preview
#69> will run only on NVIDIA GPUs for the preview I wonder why Microsoft helps nVidia, instead of using their own technology? Here’s an example: https://github.com/Const-me/Cgml
Their own technology is DirectML, not raw DirectX 11.
https://blogs.windows.com/windowsdeveloper/2023/11/15/elevat...
https://blogs.windows.com/windowsdeveloper/2023/12/14/direct...