71–80 of 175 posts
Suppose I’ve written code that calls the OpenAI API. Is there some library that helps me easily switch to a local/other LLM. I.e a library that (ideally) provides the same OpenAI interface for several models, or if not then at least the same interface.
Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our discord, so feel free to join!!
Thanks for the great project! Any chance, your team might consider more open platform than Discord for posting updates? I personally find Discord hard to use, and there’s no way to have sensible subscription (like RSS). Discord is usually muted.
To use Discord in good faith and with open eyes, you have to prioritize communication in the present, and give up hope of archiving anything that was said for people who might need the information in the future.
Earlier quoted context omitted.
Thanks for the great project! Any chance, your team might consider more open platform than Discord for posting updates? I personally find Discord hard to use, and there’s no way to have sensible subscription (like RSS). Discord is usually muted.
Discord is a black hole where information goes to die. Its search and scrollback is awful. It's awful at being an archive, as finding anything that was asked more than a day or two ago is impractical. To use Discord in good faith and with open eyes, you have to prioritize communication in the present, and give up hope of archiving anything that was said for people who might need the information in the future.
Suppose I’ve written code that calls the OpenAI API. Is there some library that helps me easily switch to a local/other LLM. I.e a library that (ideally) provides the same OpenAI interface for several models, or if not then at least the same interface.
Earlier quoted context omitted.
Discord is a black hole where information goes to die. Its search and scrollback is awful. It's awful at being an archive, as finding anything that was asked more than a day or two ago is impractical. To use Discord in good faith and with open eyes, you have to prioritize communication in the present, and give up hope of archiving anything that was said for people who might need the information in the future.
I find their search amazing. What's your issue with it?
Earlier quoted context omitted.
So all knowledge is lost and questions have to be asked and answered again and again?
That didn't stop IRC being popular in the 1990s. There has long been a place in the ecosystem for ephemeral chat. Often alongside non-ephemeral things like written documentation.
IRC chats, especially in opensource projects channels, could and would be archived, published over the web and indexed by search engines.
Earlier quoted context omitted.
That didn't stop IRC being popular in the 1990s. There has long been a place in the ecosystem for ephemeral chat. Often alongside non-ephemeral things like written documentation.
> That didn't stop IRC being popular in the 1990s. IRC chats, especially in opensource projects channels, could and would be archived, published over the web and indexed by search engines.
What kind of hardware do I need to run something small scale (1 user concurrently) and get reasonable result? Are we talking about Raspberry Pi, Core i5, Geforce 4090?
Just keep in mind the speed differences between the types of memory. If you've got a 170bn 4-bit parameter model, and you're on virtual memory on a 400 Mbps port, a naive calculation says it will take at best 28 minutes per token unless your architecture lets you skip loading parts of the model. Might take longer if the network has an internal feedback loop in the structure.