51–60 of 94 posts
Earlier quoted context omitted.
Is the NAS exposed to the whole internet? Or did you find a clever way to get CloudFlare in front of it despite it just being local?
You can use CloudFlare Tunnel ( https://www.cloudflare.com/products/tunnel/ ) to connect a system to your cloudflare gateway, without exposing it to the Internet.
Earlier quoted context omitted.
All the self-hosted LLM and text-to-image models come with some restrictions trained into them [1]. However there are plenty of people who have made uncensored "forks" of these models where the restrictions have been "trained away" (mostly by fine-tuning). You can find plenty of uncensored LLM models here: https://ollama.com/library [1]: I personally suspect that many LLMs are still trained on WebText, derivatives of…
My to-go test for uncensoring is to ask the LLM to write erotic novel. But I haven't yet find any "uncensored" ones (on ollama) that works. Did I miss something? (On the contrary: when ChatGPT first came out, it was trivial to jailbreak it to make it write erotica.)
For the people who self-host LLMs at home: what use cases do you have? Personally, I have some notes and bookmarks that I'd like to scrape, then have an LLM summarize, generate hierarchical tags, and store in a database. For the notes part at least, I wouldn't want to give them to another provider; even for the bookmarks, I wouldn't be comfortable passing my reading profile to anyone.
Earlier quoted context omitted.
All the self-hosted LLM and text-to-image models come with some restrictions trained into them [1]. However there are plenty of people who have made uncensored "forks" of these models where the restrictions have been "trained away" (mostly by fine-tuning). You can find plenty of uncensored LLM models here: https://ollama.com/library [1]: I personally suspect that many LLMs are still trained on WebText, derivatives of…
My to-go test for uncensoring is to ask the LLM to write erotic novel. But I haven't yet find any "uncensored" ones (on ollama) that works. Did I miss something? (On the contrary: when ChatGPT first came out, it was trivial to jailbreak it to make it write erotica.)
Earlier quoted context omitted.
So 8b is really smart enough to write scripts for you? How often does it fail?
> So 8b is really smart enough to write scripts for you? Depends on the model, but in general, no. ...but it's fine for simple 1 liner commands like "how do I revert my commit?" or "rename these files to camelcase". > How often does it fail? Immediately and constantly if you ask anything hard. An 8b model is not chat-gpt. The 3B model in the OP post is not chat-gpt. The capability compared to sonnet/4o is like a pota…
I’m curious about how good the performance with local LLMs is on ‘outdated’ hardware like the author’s 2060. I have a desktop with a 2070 super that it could be fun to turn into an “AI server” if I had the time…
I use a Tesla P4 for ML stuff at home, it's equivalent to a 1080 Ti, and has a score of 7.1. A 2070 (they don't list the "super") is a 7.5.
For reference, 4060 Ti, 4070 Ti, 4080 and 4090 are 8.9, which is the highest score for a gaming graphics card.
I love Coolify, used to use v3, anyone know how their v4 is going? I thought it was still a beta release from what I saw on GitHub.
ai generated blog post (or reworded, whatever) are kinda getting very irritating, like playing chess against the computer, it feel soulless