91–100 of 157 posts
“Deep understanding of what is a computer, what is computer software, and how the two relate.”
Right after the senior ML role that requires people understand how to write “algorithms and programs.”
Kinda hard to take those kinds of requirements seriously.
After the latest Chatgpt debacles, the poor performance I'm getting from 4 turbo, I'd really like a local version of chatgpt4 or equivalent. I'd even buy a new pc if I had too.
Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…
Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…
Earlier quoted context omitted.
Thanks for the link. I expected it to not let me run this. I have an intel Macbook, was expecting that I'd need Apple Silicon... am I misunderstanding something? I get fairly fast results at the prompt with the default model. How's this thing running with whatever shitty GPU I have in my laptop?
that's the magic of llama.cpp! I include a universal binary of llama.cpp's server example to do inference. What's your machine? The lowest spec I've heard it running on is a 2017 iMac with 8GB RAM (~5.5 tokens/s). On my m1 with 64GB RAM I get ~30 tokens per second on the default 7B model.
I'm now delving into getting this running in Terminal... there are a few things I want to try that I don't think the simple interface allows.
Also, I've noticed that when chats get a few kilobytes long, it just seizes up and can't go further. I complained to it, it spent a sentence apologizing, started up where it left off... and got about 12 words further.
Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…
Honest question from someone new to exploring and using these models; why do you need uncensored? What are the use-cases that would call for it? Again, not questioning your motives or anything, just straight up curious. To use your example, any of us can find bomb building info online fairly easily, and has been a point of social contention since the Anarchist's cookbook. Nobody needs an uncensored LLM for that, of c…
Earlier quoted context omitted.
Ollama does this. I run it in a container on my homelab (Proxmox on a HP EliteDesk SFF G2 800) and 7B models run decently fast on CPU-only. Ollama has a nice API and makes it easy to manage models. Together with ollama-webui, it can replace ChatGPT 3.5 for most tasks. I also use it in VSCode and nvim with plugins, works great! I have been meaning to write a short blog post about my setup...
If I may ask, which plugins are you using in VSCode?
The setup of connecting to Ollama is a bit clunky, but once it's set up it works well!
Earlier quoted context omitted.
that's the magic of llama.cpp! I include a universal binary of llama.cpp's server example to do inference. What's your machine? The lowest spec I've heard it running on is a 2017 iMac with 8GB RAM (~5.5 tokens/s). On my m1 with 64GB RAM I get ~30 tokens per second on the default 7B model.
Macbook Pro 2020 with 16gb of system ram. I think the gpu is Iris Plus? But I don't much keep up on those. I'm now delving into getting this running in Terminal... there are a few things I want to try that I don't think the simple interface allows. Also, I've noticed that when chats get a few kilobytes long, it just seizes up and can't go further. I complained to it, it spent a sentence apologizing, started up where…
Thanks for trying it!
Curious about this and I just download it. Want to try uncensored models. I have a question, looking for the most popular "uncensored" model I just find "TheBloke/Luna-AI-Llama2-Uncensored-GGML", but it has 14 files to download between 2 to 7 GB, I just download the first one: https://imgur.com/a/DE2byOB I try the model and it works: https://imgur.com/a/2vtPcui I should download all the 14 files to get better results…
I don't mean this as a criticism, I'm just curious because I work in this space too: who is this for? What is the niche of people savvy enough to use this who can't run one of the many open source local llm software? It looks in the screenshot like it's exposing much of the complexity of configuration anyway. Is the value in the interface and management of conversation and models? It would be nice to see info or even…
Being able to swap out models is also handy. This probably saved a couple of hours of my life, which I appreciate.
Amusing qualifications for the senior engineering roles they’re hiring for: “Deep understanding of what is a computer, what is computer software, and how the two relate.” Right after the senior ML role that requires people understand how to write “algorithms and programs.” Kinda hard to take those kinds of requirements seriously.
They seem to have even lowered expectations a bit. Two months ago [1] they were already hiring for that role (or a very, very similar one), but back then you needed experience with "mission-critical code in C++17", now just "production code in C++14".
1: http://web.archive.org/web/20230922170941/https://lmstudio.a...