Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenLLM

github.com

Post reply on HN
Print view View original

111–120 of 175 posts

Re: OpenLLM

#111

Post by frodowtf »

leetharris wrote: post #87
DrNosferatu wrote: post #83

The project seems great! However, newcomers (like me) are pretty blind about minimum system requirements. Could you please add them to the models list? For example: what minimum hardware do I need to run Falcon-40b? PS: If you only have a few setups "known to work" (or just one), listing that would be helpful too.

For falcon 40b you probably need an A100 40gb or so. Every model is drastically different. If you want to run something on consumer hardware, your best bet is using anything ported to the ggml framework, especially if you're on Apple silicon.

Do you know any "standard" way or measures to determine the approximate hardware requirements of a model?

Re: OpenLLM

#112

Post by [deleted] »

andsoitis wrote: post #97
DragonStrength wrote: post #95

Earlier quoted context omitted.

You might want to glance again because all of those results are for a different product.

Top of the results page says: "Showing results for how to use openlm Search instead for how to use openllm"

[deleted]

Re: OpenLLM

#113

Post by pbmonster »

paxys wrote: post #79
darkwater wrote: post #78

Earlier quoted context omitted.

> That didn't stop IRC being popular in the 1990s. IRC chats, especially in opensource projects channels, could and would be archived, published over the web and indexed by search engines.

So just like Discord then..

I wasn't aware that was being done.

Can you show me how to access the archives of the ask-for-help channel on the openllm Discord server? Right now they're discussing "loading models on CPU vs GPU". No matter how explicit I got, google did not find the discussion.

Re: OpenLLM

#114

Post by khimaros »

user432678 wrote: post #26
aarnphm wrote: post #12

Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our discord, so feel free to join!!

Thanks for the great project! Any chance, your team might consider more open platform than Discord for posting updates? I personally find Discord hard to use, and there’s no way to have sensible subscription (like RSS). Discord is usually muted.

plugging the open source and self hostable https://revolt.chat which i've found to have great UX and be very performant compared to discord.

Re: OpenLLM

#115

Post by numpad0 »

pmontra wrote: post #63
cinntaile wrote: post #47

Earlier quoted context omitted.

Because it's easy, free and it just works. Very few people actually care about indexing the conversations.

So all knowledge is lost and questions have to be asked and answered again and again?

No, questions don't have to be asked and answered again and again, because all the knowledge is lost, full stop. No one would know anything.

Re: OpenLLM

#116

Post by falcor84 »

scubbo wrote: post #106
falcor84 wrote: post #81

Earlier quoted context omitted.

Isn't there something really nice about it though? It seems to me that most every community gradually evolves into one where every new message from a new-ish member is answered by something like "Duplicate, please search first!". And this in turn makes those newcomers either go away, become passive lurkers, or become part of the "hive-mind" (as only likeminded questions get answered). On the other hand, if people hav…

The best of both worlds - a friendly community that welcomes newbies, with a searchable archive - is possible. Limiting to only chat-based support means that support is bottle-necked by the folks who are available and engaged at the time of the question, and that knowledge will "drop out" of the community as people forget it.

Apologies for my skepticism, but is it just "possible", or do you actually have an example of a long-lived community that remained fully welcoming to newbies while utilizing a searchable archive?

In any case, I'm not arguing that it's impossible, but rather that the more comprehensive the archive, the less welcoming the community would tend to be, all other things being equal. To take it to the extreme, I'll posit the following law: "A well-curated archive is the grave of a community"

Re: OpenLLM

#118

Post by dunefox »

Zak wrote: post #85
decide1000 wrote: post #51

Earlier quoted context omitted.

Isn't it a generation thing? If I had the choice everyone would be on IRC still.

I've used IRC for a long time and still do, but I do think Discord has a nicer UX for most use cases. In particular, building communities around clusters of channels ("servers") and support for rich media (yes, some old people might call that a downside) increase the appeal for most people. It's also a lot more work to have a persistent connection on IRC (bouncers). My main problem with Discord is that it's someone e…

Matrix would be my choice as well but good luck getting people to use the uglier alternative. Discord is great.

Re: OpenLLM

#119

Post by winddude »

pmoriarty wrote: post #72
user432678 wrote: post #26

Earlier quoted context omitted.

Thanks for the great project! Any chance, your team might consider more open platform than Discord for posting updates? I personally find Discord hard to use, and there’s no way to have sensible subscription (like RSS). Discord is usually muted.

Discord is a black hole where information goes to die. Its search and scrollback is awful. It's awful at being an archive, as finding anything that was asked more than a day or two ago is impractical. To use Discord in good faith and with open eyes, you have to prioritize communication in the present, and give up hope of archiving anything that was said for people who might need the information in the future.

agreed.

Re: OpenLLM

#120

Post by numpad0 »

Though only related to topic wrt LLMs: It seems LLMs occasionally mix up CJK vocabularies and also generate invalid UTF-8 sequences, due to CJK texts having overlapping code points and inputs being processed by tokenizer. Are there developments in that directions? Aren't CJK ideograms essentially tokens?
Post reply on HN