Which graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.
It’s about VRAM, I would say the more the better, 4060 with 8GB should be the starting point
Run Llama 2 uncensored locally
11–20 of 225 posts
Re: Run Llama 2 uncensored locally
#12Finally, I was just trying to find similar models yesterday! Seems like "evil" was not the correct keyword .)
Re: Run Llama 2 uncensored locally
#13Which graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.
The GPUs with the most VRAM you can justify spending money on.
Re: Run Llama 2 uncensored locally
#14Which graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.
It’s about VRAM, I would say the more the better, 4060 with 8GB should be the starting point
3060 with 12gb is cheaper and provides more vram.
Re: Run Llama 2 uncensored locally
#15Modelfile cool. Just use JSON or YAML, not custom format please! edit: Adapter support would be really cool. Multiple adapter even better i want somebody to make MoE of adapters.
It's modeled after Dockerfiles and is pretty easy to understand due to that. I don't know what advantage a more complex format would bring.
i know, its not about complexity of format. somebody is going run into problem where they have to parse modelfile and parser is not written in their language vs it could be done with yaml/json/toml. Its just me i guess.
ziglang is adding package manager, and they decided to roll own `zon` format or sth which is bashed on their language struct syntax. i do not like it. i would not say never custom DSL formats, but most of the time they are overkill.
{.abc="123"}
Re: Run Llama 2 uncensored locally
#16Which graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.
A 16GB 3080 is probably the cheapest and most ideal in a big laptop.
But you can get some acceleration with anything ~6GB and up.
Re: Run Llama 2 uncensored locally
#17Re: Run Llama 2 uncensored locally
#18Which graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.
Also, what size and ballpark price are you looking for?
Re: Run Llama 2 uncensored locally
#19Re: Run Llama 2 uncensored locally
#20I had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.