Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PrivateGPT

github.com

Post reply on HN
Print view View original

101–110 of 147 posts

Re: PrivateGPT

#101

Post by deadeye »

TeMPOraL wrote: post #75
flagrant_taco wrote: post #18

Earlier quoted context omitted.

What's the point of the technology if it will provide an answer regardless of the accuracy? And what prevents this from being dangerous when the factual and ficticious answers are indistinguishable?

We have the same problem with people . Somehow, we've managed to build a civilization that can, occasionally, fly people to the Moon and get them back. Even if LLMs never get any more reliable than your average human, they're still valuable because they know much more than any single human ever could, run faster, only eat electricity, and can be scaled up without all kinds of nasty social and political problems. That…

"without all kinds of nasty social and political problems"

I assure you, those still exist in AI. AI follows whatever political dogma it is trained on, regardless of if you point out how logically flawed it is.

If it is trained to say 1+1=3, then no matter what proofs you provide, it will not budge.

Re: PrivateGPT

#102

Post by notatallshaw »

davidy123 wrote: post #80
oefrha wrote: post #67

Earlier quoted context omitted.

You don’t need Docker, you just need a virtual env for each random thing you try instead of making them all conflict with each other. Maybe some day pip will add a switch to automatically create one, but until then, python3 -m venv venv . venv/bin/activate before you try something random. Also, `python` is usually Python 2.7. If it is, I advise removing it from your system unless you have a strong reason to keep it.

Sorry, but (base) vid@kk:~/D/ai/tortoise-tts$ python3 -m venv venv (base) vid@kk:~/D/ai/tortoise-tts$ . venv/bin/activate (venv) (base) vid@kk:~/D/ai/tortoise-tts$ python -m pip install -r ./requirements.txt Collecting tqdm Using cached tqdm-4.65.0-py3-none-any.whl (77 kB) Collecting rotary_embedding_torch Using cached rotary_embedding_torch-0.2.3-py3-none-any.whl (4.5 kB) … × python setup.py egg_info did not run suc…

`rotary_embedding_torch` has not defined any build requirements hence your error: https://github.com/lucidrains/rotary-embedding-torch. You therefore need to install `numpy` before installing `rotary_embedding_torch`.

This is bad, `rotary_embedding_torch` as a package is not in a high enough quality to put as a requirement.

The good news is Pip 23.1+ is forcing the issue, `rotary_embedding_torch` will fail even if you have `numpy` installed because builds by default take place in an isolated environment and you *must* define any build requirements you have. This should force the quality of packages in the Python ecosystem to improve and no longer have this error.

Re: PrivateGPT

#103

Post by oceanplexian »

davidy123 wrote: post #80
oefrha wrote: post #67

Earlier quoted context omitted.

You don’t need Docker, you just need a virtual env for each random thing you try instead of making them all conflict with each other. Maybe some day pip will add a switch to automatically create one, but until then, python3 -m venv venv . venv/bin/activate before you try something random. Also, `python` is usually Python 2.7. If it is, I advise removing it from your system unless you have a strong reason to keep it.

Sorry, but (base) vid@kk:~/D/ai/tortoise-tts$ python3 -m venv venv (base) vid@kk:~/D/ai/tortoise-tts$ . venv/bin/activate (venv) (base) vid@kk:~/D/ai/tortoise-tts$ python -m pip install -r ./requirements.txt Collecting tqdm Using cached tqdm-4.65.0-py3-none-any.whl (77 kB) Collecting rotary_embedding_torch Using cached rotary_embedding_torch-0.2.3-py3-none-any.whl (4.5 kB) … × python setup.py egg_info did not run suc…

FYI Tortoise, the thing you are trying to build, is abandonware. The creator decided to stop working on it due to “ethics” (i.e only Big Tech should have access to AI) when the community reverse engineered a way to finetune it using weights accidentally left on hugging face. There’s a nice fork out there called mrq/ai-voice-cloning.

Re: PrivateGPT

#104

Post by davidy123 »

oceanplexian wrote: post #103
davidy123 wrote: post #80

Earlier quoted context omitted.

Sorry, but (base) vid@kk:~/D/ai/tortoise-tts$ python3 -m venv venv (base) vid@kk:~/D/ai/tortoise-tts$ . venv/bin/activate (venv) (base) vid@kk:~/D/ai/tortoise-tts$ python -m pip install -r ./requirements.txt Collecting tqdm Using cached tqdm-4.65.0-py3-none-any.whl (77 kB) Collecting rotary_embedding_torch Using cached rotary_embedding_torch-0.2.3-py3-none-any.whl (4.5 kB) … × python setup.py egg_info did not run suc…

FYI Tortoise, the thing you are trying to build, is abandonware. The creator decided to stop working on it due to “ethics” (i.e only Big Tech should have access to AI) when the community reverse engineered a way to finetune it using weights accidentally left on hugging face. There’s a nice fork out there called mrq/ai-voice-cloning.

Thanks, I didn't know that backstory. I will check out that repo.

Re: PrivateGPT

#105

Post by arbol »

superbiome wrote: post #70

Does something like this exist for local code repos? (Excuse my ignorance since the space is moving faster than light.)

Seems to me that this could be used for exactly that. Just fork the repo and change the filetypes and loaders for your code source files.

Re: PrivateGPT

#107

Post by kseudo »

soferio wrote: post #17
thefourthchime wrote: post #16

I tried this on my M2 Macbook with 16gb of RAM but got: "ggml_new_tensor_impl: not enough space in the context's memory pool (needed 18296202768, available 18217606000)"

Anyone got it working on an M1 with 8gb?

I got it working on an M1 with 16gb. Quite slow but it trains and returns responses.

Re: PrivateGPT

#108

Post by lambda_matt »

btbuildem wrote: post #97
oefrha wrote: post #67

Earlier quoted context omitted.

You don’t need Docker, you just need a virtual env for each random thing you try instead of making them all conflict with each other. Maybe some day pip will add a switch to automatically create one, but until then, python3 -m venv venv . venv/bin/activate before you try something random. Also, `python` is usually Python 2.7. If it is, I advise removing it from your system unless you have a strong reason to keep it.

Nope, this is EXACTLY why I'd use docker. You want to faff around with some esoteric settings? Go for it! But don't make your would-be users runt the gauntlet, that's pointless. All that nonsense vs docker pull / docker run

docker is not the right tool for the job here. this is not an app. this is a nacent project and if you want people to benefit from the underlying code, and contribute back to it to grow this field, you provide proof of concept code, not full, complex and opinionated interfaces that are all crufted up with containerization/packaging. venv is a core module of python and its dead simple to get a virtual environment up and running. you dont have to do any crazy things to expose hardware to it (GPUS), you just run two commands to create and source the environment and then everything just works.

Re: PrivateGPT

#109

Post by Nephx »

oefrha wrote: post #67
davidy123 wrote: post #65

Granted I'm not coming from the python world, but I have tried many of these projects, and very few of them install out of the box. They usually end with some incompatibility, and files scattered all over the place, leading to future nightmares. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.…

You don’t need Docker, you just need a virtual env for each random thing you try instead of making them all conflict with each other. Maybe some day pip will add a switch to automatically create one, but until then, python3 -m venv venv . venv/bin/activate before you try something random. Also, `python` is usually Python 2.7. If it is, I advise removing it from your system unless you have a strong reason to keep it.

Dangerous comment.

From a linux perspective, I wouldn't blindly suggest the average reader to purge Python 2.7 from their system, as it might drag core parts of the WM with it. Consider aliasing, or better yet, relying on modern venv tools such as Conda instead.

Re: PrivateGPT

#110

Post by subsaharancoder »

oefrha wrote: post #67
davidy123 wrote: post #65

Granted I'm not coming from the python world, but I have tried many of these projects, and very few of them install out of the box. They usually end with some incompatibility, and files scattered all over the place, leading to future nightmares. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.…

You don’t need Docker, you just need a virtual env for each random thing you try instead of making them all conflict with each other. Maybe some day pip will add a switch to automatically create one, but until then, python3 -m venv venv . venv/bin/activate before you try something random. Also, `python` is usually Python 2.7. If it is, I advise removing it from your system unless you have a strong reason to keep it.

+1 to using venv
Post reply on HN