61–70 of 78 posts
Earlier quoted context omitted.
https://github.com/jmorganca/ollama/tree/main/examples/priva... there's an example using PrivateGPT too
Is it private and offline via ollama? Are all ollama models private and offline?
I didn't see any info on how this is different than installing/running llamacpp or koboldcpp. New offerings are awesome of course, but what is it adding?
The main difference is setting everything up yourself manually, downloading the modal, optimizing the parameters for best performance, running an API server and a UI front-end - which is out of reach for most non-technical people. With LlamaGPT, it's just one command: `docker compose up -d` or one click install for umbrelOS home server users.
Gpt4all[1] offers a similar 'simple setup' but with application exe downloads, but is arguably more like open core because the gpt4all makers (nomic?) want to sell you the vector database addon stuff on top.
[1]https://github.com/nomic-ai/gpt4all
I like this one because it feels more private / is not being pushed by a company that can do a rug pull. This can still do a rug pull, but it would be harder to do.
Earlier quoted context omitted.
The main difference is setting everything up yourself manually, downloading the modal, optimizing the parameters for best performance, running an API server and a UI front-end - which is out of reach for most non-technical people. With LlamaGPT, it's just one command: `docker compose up -d` or one click install for umbrelOS home server users.
Maybe I've been at this for too long and can't see the pitfalls of a normal user, but how is that easier than using an oobabooga one-click installer (an option that's been around "forever")? I guess ooba one-click doesn't come with a model included, but is that really enough of a hurdle to stop someone from getting it going? Maybe I'm not seeing the value proposition of this. Glad to be enlightened!
HN users (mostly) don't actually read or check anything and upvote mostly based on titles and subsequent early comments.
Earlier quoted context omitted.
Well, GPT is simply an initialism for "Generative Pre-trained Transformer". In Germany, a trademark can be lost if it becomes a "Gattungsbegriff" (generic term). This happens when a trademark becomes so well-known and widely used that it becomes the common term for a product or service, rather than being associated with a specific company or brand. For example, if a company invented a new type of vacuum cleaner and t…
Actually in the UK and Ireland a vacuum cleaner is called a Hoover. But in general I think we do that less than Americans. For example, we don't call a public announcement system a "Tannoy". That's a brand of hifi speakers. And we'd say "photo copier" instead of Xerox.
Very cool, this looks like a combination of chatbot-ui and llama-cpp-python? A similar project I've been using is https://github.com/serge-chat/serge . Nous-Hermes-Llama2-13b is my daily driver and scores high on coding evaluations ( https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul... ).
Earlier quoted context omitted.
The simplest way, as rdedev is describing, is to do Retrieval Augmented Generation (RAG) in your prompting. This would require the addition of a vector database and a text embedding model. There are many open source / local / private options for that. The steps would then be: 1. Embed your private data in chunks and store the resulting embeddings in a vector database 2. In your prompting workflow, when a user queries…
Looks like this is not easy at all for a non ML expert. And probably the required computing power is still out of reach for mere mortals. I'd have a similar use case to the parent: technical books. I'd love to be able to ask where a certain topic is discussed in my pdf archive and have the AI reply with references and possibly a significant piece of the relevant articles, with images (or local links to them).
Earlier quoted context omitted.
Actually in the UK and Ireland a vacuum cleaner is called a Hoover. But in general I think we do that less than Americans. For example, we don't call a public announcement system a "Tannoy". That's a brand of hifi speakers. And we'd say "photo copier" instead of Xerox.
You don't call a PA system a tannoy? That's news to all my English friends, who keep telling me that people announced things over the tannoy.
Earlier quoted context omitted.
The simplest way, as rdedev is describing, is to do Retrieval Augmented Generation (RAG) in your prompting. This would require the addition of a vector database and a text embedding model. There are many open source / local / private options for that. The steps would then be: 1. Embed your private data in chunks and store the resulting embeddings in a vector database 2. In your prompting workflow, when a user queries…
Yeah but the big question I kept having and missing the answer is: How do you encode the private data into the vectors? It is a bunch of text but how do you choose the vector values in the first place? What software does that? Isn’t that basically an ML task with its own weights, that’s what classifiers do! I was surprised everyone had been writing about that but neglecting to explain this piece. Like math textbooks…
How do you know that Claude doesn't do this? If you have multiple books, you end up with more than 100k context, and running the model with full context takes more time so it is more expensive as well.