Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PrivateGPT

github.com

Post reply on HN
Print view View original

131–140 of 147 posts

Re: PrivateGPT

#131

Post by jeadie »

ChocoluvH wrote: post #50
kacperlukawski wrote: post #48

Earlier quoted context omitted.

Chroma doesn't seem to be a real DB, it's rather a wrapper around tools like hnswlib, DuckDB or Clickhouse. Qdrant is way more mature - it has its own HNSW implementation with some tweaks to incorporate filtering directly during the vector search phase, supports horizontal and vertical scaling, as well as provides its own managed cloud offering. In general, Qdrant is a real DB, not a library and that's a huge differe…

What does Chroma lack? Their APIs seem pretty much the same to me.

I've tried both Chroma and Qdrant. I don't think Chroma lacks that much. Definitely newer, but is also a great product. I think cloud support coming Q3 2023

Re: PrivateGPT

#133

Post by pmarreck »

bspammer wrote: post #128
pmarreck wrote: post #93

Earlier quoted context omitted.

> the ecosystem does has a packaging reproducibility and multitenancy problem with out-of-box tooling this is exactly why I am learning Nix, to help contain chaotically-designed dependency garbage like this to 1 project directory

Yep nix is awesome at this kind of thing. Check out this project which packages a couple of AI projects with nix, both work out of the box for me. https://nixified.ai/

oh absolutely YES.

Ironically, the poo of things like python multitenant dependency management will likely push Nix adoption forward (and unfortunately also Docker)

Re: PrivateGPT

#134

Post by mewpmewp2 »

oefrha wrote: post #85
davidy123 wrote: post #80

Earlier quoted context omitted.

Sorry, but (base) vid@kk:~/D/ai/tortoise-tts$ python3 -m venv venv (base) vid@kk:~/D/ai/tortoise-tts$ . venv/bin/activate (venv) (base) vid@kk:~/D/ai/tortoise-tts$ python -m pip install -r ./requirements.txt Collecting tqdm Using cached tqdm-4.65.0-py3-none-any.whl (77 kB) Collecting rotary_embedding_torch Using cached rotary_embedding_torch-0.2.3-py3-none-any.whl (4.5 kB) … × python setup.py egg_info did not run suc…

Oh I don't disagree, the ecosystem does has a packaging reproducibility and multitenancy problem with out-of-box tooling, and projects seldom provide basic instructions for people outside the ecosystem, like using a virtual env. That said, this tortoise-tts project might be a particularly bad example. It somehow locks to scipy 0.10.1 from 2012 [1] (during the Python 3.2 release cycle, when Python 3 was heavily in flu…

I think I personally keep running into these bad examples every time I use something with Python and I do use venv every time. Rarely something works out of the box. Even colabs I try somehow won't work after a while. There is always some sort of version mismatch, sometimes something like numpy, tensorflow and some other deps.

Re: PrivateGPT

#135

Post by udev4096 »

Mystery-Machine wrote: post #121
udev4096 wrote: post #111

Earlier quoted context omitted.

People get very easily offended these days. What's wrong with that tone? I was just simply stating a fact

You are right. People get easily offended. But your tone does seem a bit _freaking_ upset. Over what? The fact that someone posted the same link like you did and got more clicks from random users browsing the internet? Is that what upset you? What do you get if you got billions of clicks and upvotes? They say that people who need external validation don't have their own values and need to seek approval from others...

I certainly do not care about that. I shouldn't have mentioned it in the first place. Apologies from my side

Re: PrivateGPT

#136

Post by weikju »

unstatusthequo wrote: post #132
weikju wrote: post #41

Earlier quoted context omitted.

Is it private if it's using ChatGPT?

They will say using API means you data isn’t used for training. True if you believe OpenAI t&c’s. But that’s different than being able to say you didn’t send data to any third party.

Indeed, it's still a far cry from being private if the data is leaving my device for any reason.

Re: PrivateGPT

#137

Post by rolisz »

moffkalast wrote: post #120
rolisz wrote: post #76

Earlier quoted context omitted.

Well you wouldn't input the whole Vault to the model, you would use embeddings to find the content that is most relevant to the question being asked.

Is that actually a thing yet? Proper vector DB integration? I sure would like to see some demos of that, as it's been hyped up a lot but I haven't really seen anyone deploy anything proper with it yet.

Even PrivateGPT does that, using Chroma as vector DB

Re: PrivateGPT

#138

Post by johnnyjeans »

davidy123 wrote: post #65

Granted I'm not coming from the python world, but I have tried many of these projects, and very few of them install out of the box. They usually end with some incompatibility, and files scattered all over the place, leading to future nightmares. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.…

This is the primary reason I'm averse to languages and ecosystems that rely on package managers. I have never had a good experience where these things aren't just constantly breaking. Stack/cabal, cargo, pip, npm/yarn, gem. Scattering files across my filesystem and having extremely brittle configs that shatter the ecosystem into a billion pieces at seemingly random intervals. A problem exacerbated by these package managers often being more complex than the compiler/interpreter itself. Luarocks is probably the least problematic, and that's mostly because it hosts really simple and self-contained software.

Say what you will about the old school way of manually building and copying shit around, at least when something breaks I don't have to spend a couple hours keelhauling a bloated toolchain in a debugger for mutiny.

Re: PrivateGPT

#139

Post by gymbeaux »

oefrha wrote: post #67
davidy123 wrote: post #65

Granted I'm not coming from the python world, but I have tried many of these projects, and very few of them install out of the box. They usually end with some incompatibility, and files scattered all over the place, leading to future nightmares. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.…

You don’t need Docker, you just need a virtual env for each random thing you try instead of making them all conflict with each other. Maybe some day pip will add a switch to automatically create one, but until then, python3 -m venv venv . venv/bin/activate before you try something random. Also, `python` is usually Python 2.7. If it is, I advise removing it from your system unless you have a strong reason to keep it.

It’s been a long time since “Python —version” output “2.x” on a computer I was using. Even macOS is on Python 3 these days iirc. Every Linux distro I’ve installed in the last few months was at least 3.8.

Re: PrivateGPT

#140

Post by zestyping »

Would someone do me the kindness of explaining (a little more) how this works?

It looks like you can ask a question and the model will use its combined knowledge of all your documents to figure out the answer. It looks like it isn't fine-tuned or trained on all the documents, is that right? How is each document turned into an embedding, and then how does the model figure out which documents to consult to answer the question?

Post reply on HN