Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PrivateGPT

github.com

Post reply on HN
Print view View original

121–130 of 147 posts

Re: PrivateGPT

#121

Post by Mystery-Machine »

udev4096 wrote: post #111
andruby wrote: post #57

Earlier quoted context omitted.

possibly because the tone of your post and that it doesn't actually add to the conversation. weekday and time of day have an impact. Thousands of entries are posted each day (see https://news.ycombinator.com/newest ) most never get a comment or upvote.

People get very easily offended these days. What's wrong with that tone? I was just simply stating a fact

You are right. People get easily offended. But your tone does seem a bit _freaking_ upset. Over what? The fact that someone posted the same link like you did and got more clicks from random users browsing the internet? Is that what upset you? What do you get if you got billions of clicks and upvotes?

They say that people who need external validation don't have their own values and need to seek approval from others...

Re: PrivateGPT

#122

Post by cubefox »

quaintdev wrote: post #81
cddotdotslash wrote: post #72

Earlier quoted context omitted.

I suspect the major cloud providers will also each offer their own “enterprise friendly” LLM services (Azure already offers a version of OpenAI’s API). If they have the right data guarantees, that’ll probably be sufficient for companies that are already using their IaaS offerings.

Enterprises should work on an open source LLM and run it on their own. This also helps people like you and me to run LLM at home. It has worked before like in case of Linux and can work again.

Powerful LLMs are so large that they can only be trained by the major AI companies. Even LLaMA 65B (where the open release was less than intended) can't compete with GPT-3.5, let alone GPT-4. And the price for the most powerful models will only increase now, as we have effectively an arms race between OpenAI/Microsoft and Google. Few, if anyone, will be able to keep up.

Linux is different. It doesn't require huge investments in server farms.

Re: PrivateGPT

#123

Post by zora_goron »

simonw wrote: post #2

I'm always interested in seeing the prompt that drives these kinds of tools. In this case it appears to be using RetrievalQA from LangChain, which I think is this prompt here: https://github.com/hwchase17/langchain/blob/v0.0.176/langcha... Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. {context} Questio…

On a related note, in case it's of interest to anyone else -- I pulled out all the default prompts from LangChain and put them up here: https://github.com/samrawal/langchain-prompts/blob/main/READ...

Re: PrivateGPT

#124

Post by ShadowBanThis01 »

davidy123 wrote: post #65

Granted I'm not coming from the python world, but I have tried many of these projects, and very few of them install out of the box. They usually end with some incompatibility, and files scattered all over the place, leading to future nightmares. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.…

Yep, I just tried to install a Python-based project and there was a conflict between Pyenv's and Homebrew's versions of pip... despite having used Homebrew to install Pyenv. I ended up just getting rid of Pyenv altogether... but now Python may be in some screwed-up state on my system.

It's too bad the ecosystem seems to be so messy, because Python seems like the best language for general utilities.

Re: PrivateGPT

#125

Post by lrdd »

cubefox wrote: post #122
quaintdev wrote: post #81

Earlier quoted context omitted.

Enterprises should work on an open source LLM and run it on their own. This also helps people like you and me to run LLM at home. It has worked before like in case of Linux and can work again.

Powerful LLMs are so large that they can only be trained by the major AI companies. Even LLaMA 65B (where the open release was less than intended) can't compete with GPT-3.5, let alone GPT-4. And the price for the most powerful models will only increase now, as we have effectively an arms race between OpenAI/Microsoft and Google. Few, if anyone, will be able to keep up. Linux is different. It doesn't require huge inv…

I think you would be interested in Google's internal memo[0] that did the rounds here a couple weeks ago. The claim is that OpenAI and all competition is destined to fall behind open-source. All you need is a big model to be released and all fine tuning can be done by a smart, budget, distributed workforce.

[0]: https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

Re: PrivateGPT

#126

Post by cubefox »

lrdd wrote: post #125
cubefox wrote: post #122

Earlier quoted context omitted.

Powerful LLMs are so large that they can only be trained by the major AI companies. Even LLaMA 65B (where the open release was less than intended) can't compete with GPT-3.5, let alone GPT-4. And the price for the most powerful models will only increase now, as we have effectively an arms race between OpenAI/Microsoft and Google. Few, if anyone, will be able to keep up. Linux is different. It doesn't require huge inv…

I think you would be interested in Google's internal memo[0] that did the rounds here a couple weeks ago. The claim is that OpenAI and all competition is destined to fall behind open-source. All you need is a big model to be released and all fine tuning can be done by a smart, budget, distributed workforce. [0]: https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

But why would a big model be released? LLaMA can't even begin to compete with GPT-4. Fine-tuning won't make it more intelligent. The only entity currently able to compete with OpenAI/Microsoft is Google with their planned Gemini model.

Re: PrivateGPT

#127

Post by j_shi »

londons_explore wrote: post #117
j_shi wrote: post #69

Self-hosted + self-trained LLMs are probably the future for enterprise. While consumers are happy to get their data mined to avoid paying, businesses are the opposite: willing to pay a lot to avoid feeding data to MSFT/GOOG/META. They may give assurances on data protection (even here GitHub copilot TOS has sketchy language around saving down derived data), but can’t get around fundamental problem that their products…

> willing to pay a lot to avoid feeding data to MSFT/GOOG/META. Right now, you can't pay a lot and get a local LLM with similar performance to GPT-4. Anything you can run on-site isn't really even close in terms of performance. The ability to finetune to your workplaces terminology and document set is certainly a benefit, but for many usecases that doesn't outweigh the performance difference.

Open source with own fine tuning closing in fast:

https://lmsys.org/blog/2023-03-30-vicuna/

https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

Re: PrivateGPT

#128

Post by bspammer »

pmarreck wrote: post #93
oefrha wrote: post #85

Earlier quoted context omitted.

Oh I don't disagree, the ecosystem does has a packaging reproducibility and multitenancy problem with out-of-box tooling, and projects seldom provide basic instructions for people outside the ecosystem, like using a virtual env. That said, this tortoise-tts project might be a particularly bad example. It somehow locks to scipy 0.10.1 from 2012 [1] (during the Python 3.2 release cycle, when Python 3 was heavily in flu…

> the ecosystem does has a packaging reproducibility and multitenancy problem with out-of-box tooling this is exactly why I am learning Nix, to help contain chaotically-designed dependency garbage like this to 1 project directory

Yep nix is awesome at this kind of thing. Check out this project which packages a couple of AI projects with nix, both work out of the box for me.

https://nixified.ai/

Re: PrivateGPT

#129

Post by GloriousKoji »

btbuildem wrote: post #100
oefrha wrote: post #99

Earlier quoted context omitted.

Now your user need to learn a lot about Docker to edit anything. And you need to find a place to host those huge images. What's free today may not be in a year, see Docker Hub. Edit: Not saying offering it as an option is bad. But your pip install should work regardless, scipy=0.10.1 is bad whether you offer a Docker image or not.

If you don't want to host Docker images, you can just provide a Dockerfile. That way the onus of resolving all the complications is on you - your user only needs to have Docker running on their system. Arguably, it's a pretty reasonable requirement. Widely used, mature, easy to set up. I don't remember when I switched to running all my dev envs in Docker, but I wouldn't go back.

How far are we going to need to go to fully abstract these systems? Am I going to need a separate computer running a VM with a server image for hosting a docker image of python venv to manage a package that prints some text?

Re: PrivateGPT

#130

Post by Baeocystin »

cubefox wrote: post #126
lrdd wrote: post #125

Earlier quoted context omitted.

I think you would be interested in Google's internal memo[0] that did the rounds here a couple weeks ago. The claim is that OpenAI and all competition is destined to fall behind open-source. All you need is a big model to be released and all fine tuning can be done by a smart, budget, distributed workforce. [0]: https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

But why would a big model be released? LLaMA can't even begin to compete with GPT-4. Fine-tuning won't make it more intelligent. The only entity currently able to compete with OpenAI/Microsoft is Google with their planned Gemini model.

…today. But with the amount of (justifiable, IMO) attention LLMs are now getting, I don't see how this won't change soon. And there's quite a bit of incentive for second- or third-tier companies to contribute to something that could kneecap the bigger players.
Post reply on HN