141–147 of 147 posts
Earlier quoted context omitted.
Anyone got it working on an M1 with 8gb?
I got it working on an M1 with 16gb. Quite slow but it trains and returns responses.
Using embedded DuckDB with persistence: data will be stored in: db [1] 8281 illegal hardware instruction python privateGPT.py
Earlier quoted context omitted.
I suspect the major cloud providers will also each offer their own “enterprise friendly” LLM services (Azure already offers a version of OpenAI’s API). If they have the right data guarantees, that’ll probably be sufficient for companies that are already using their IaaS offerings.
How do the data rights broadly differ between OpenAI API directly and through Azure's endpoint?
The big benefits are AAD auth and the ability to put a proxy (APIM, etc.) on the OpenAI endpoint to do quality control, metering, logging, moderation, etc. all within Azure.
Earlier quoted context omitted.
> willing to pay a lot to avoid feeding data to MSFT/GOOG/META. Right now, you can't pay a lot and get a local LLM with similar performance to GPT-4. Anything you can run on-site isn't really even close in terms of performance. The ability to finetune to your workplaces terminology and document set is certainly a benefit, but for many usecases that doesn't outweigh the performance difference.
Open source with own fine tuning closing in fast: https://lmsys.org/blog/2023-03-30-vicuna/ https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...
Earlier quoted context omitted.
If you don't want to host Docker images, you can just provide a Dockerfile. That way the onus of resolving all the complications is on you - your user only needs to have Docker running on their system. Arguably, it's a pretty reasonable requirement. Widely used, mature, easy to set up. I don't remember when I switched to running all my dev envs in Docker, but I wouldn't go back.
How far are we going to need to go to fully abstract these systems? Am I going to need a separate computer running a VM with a server image for hosting a docker image of python venv to manage a package that prints some text?
Granted I'm not coming from the python world, but I have tried many of these projects, and very few of them install out of the box. They usually end with some incompatibility, and files scattered all over the place, leading to future nightmares. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.…
Yep, I just tried to install a Python-based project and there was a conflict between Pyenv's and Homebrew's versions of pip... despite having used Homebrew to install Pyenv. I ended up just getting rid of Pyenv altogether... but now Python may be in some screwed-up state on my system. It's too bad the ecosystem seems to be so messy, because Python seems like the best language for general utilities.
Granted I'm not coming from the python world, but I have tried many of these projects, and very few of them install out of the box. They usually end with some incompatibility, and files scattered all over the place, leading to future nightmares. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.…
You should not be seeing these dependency conflict issues if you install each project in its own virtual environment.
If you just want them to be easily installed you can just use pipx (`pipx install my-package`) which will manage the virtual environment automatically.
Making a full blown Docker image for it is overkill 99% of the time. Virtual environments serve the same purpose while being much faster and lighter weight.
Granted I'm not coming from the python world, but I have tried many of these projects, and very few of them install out of the box. They usually end with some incompatibility, and files scattered all over the place, leading to future nightmares. ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.…
Yep, I just tried to install a Python-based project and there was a conflict between Pyenv's and Homebrew's versions of pip... despite having used Homebrew to install Pyenv. I ended up just getting rid of Pyenv altogether... but now Python may be in some screwed-up state on my system. It's too bad the ecosystem seems to be so messy, because Python seems like the best language for general utilities.
The comment does not really make sense. It sounds like pyenv wasn't setup correctly on your system (needs to be added to your bash_profile etc). The typical setup is to put pyenv first in PATH so it takes precedence.
It may be easier to use pipx which will auto manage the virtual envs of end user apps for you.