Do we know how LLMs available in OpenLLM and other open source LLMs compare to different versions of GPT models? I know there’s a leaderboard on huggingface: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb... but it doesn’t contain GPT models.
OpenLLM
41–50 of 175 posts
Re: OpenLLM
#42This sounds promising. Smaller but custom trained/tuned models would be ideal - works for the task without the overhead
Agreed, mostly in real life we have specialized experts. You go to doctor to ask about health related stuff, you ask your colleague who is expert in ML only about ML and probably not much about mobile development.
Instead if having ML that is expert at coding in all languages probably would be better to allow to switch context.
Native mobile dev LLM? - e.g. train only on swift, objc, kotlin, java, c, c++ code
Python dev LLM? - train only on python, c, c++, rust code
Would this way final model be smaller, faster and maybe event better?
Re: OpenLLM
#43Earlier quoted context omitted.
In my experience it doesn't work like that. It's like if you take an idiot and spend a bunch of time training them, they'll still perform much worse than a moderately intelligent person with much less training. And smaller models can be pretty idiotic.
Then why do Chess AI perform much better than LLMs trying to play chess.
Because there’s a world of difference between a reinforcement learning trained special purpose model and asking a general purpose large language model to have a go at something.
Re: OpenLLM
#44Fine-tuning is the most important part, but it is under intense research today, things change fast. I hope they can streamline this process because these smaller models can only compete with big models when they are fine-tuned.
Yes but fine tuning requires a lot more gpu memory and is thus much more expensive, complicated and out of reach of most people. To fine tune a >10B model you still need multiple A100 / H100. Let’s hope that changes with quantized fine tuning, forward pass only etc.
Can you fine tune on an M2 with adequate memory?
Re: OpenLLM
#45Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our discord, so feel free to join!!
Side question : why are people working on open source project communicating through discord a lot noawadays ?
are discord conversations persisted and indexed on search engines ?
Re: OpenLLM
#46Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our discord, so feel free to join!!
Side question : why are people working on open source project communicating through discord a lot noawadays ? are discord conversations persisted and indexed on search engines ?
1.People like talking to each other on discord.
2. No. :/
Re: OpenLLM
#47Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our discord, so feel free to join!!
Side question : why are people working on open source project communicating through discord a lot noawadays ? are discord conversations persisted and indexed on search engines ?
Because it's easy, free and it just works.
Very few people actually care about indexing the conversations.
Re: OpenLLM
#48Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our discord, so feel free to join!!
Side question : why are people working on open source project communicating through discord a lot noawadays ? are discord conversations persisted and indexed on search engines ?
short answer: because it's one of the options with least friction to get running
a lot of people who are into tech stuff already have a discord account making joining the community a one click process, the instant nature of it seems to appeal to younger users more than async forums, it's a fairly mature platform so it has a bunch of moderation/customization/integration features you might want, etc.
> are discord conversations persisted and indexed on search engines ?
nope (and that is a drawback many point out)
Re: OpenLLM
#49Does it work only with text? Or image/video processing too?
Re: OpenLLM
#50Hi all, I'm the main maintainer from the OpenLLM team here. I'm actively developing the fine-tuning feature and will release a PR soon enough. Stay tuned. In the meanwhile, the best way to track the development workflow is at our discord, so feel free to join!!
Side question : why are people working on open source project communicating through discord a lot noawadays ? are discord conversations persisted and indexed on search engines ?
I find Discord quite versatile and a bit overwhelming at the same time. As to SEO, see https://news.ycombinator.com/item?id=36383773
AFAIK most of the gamers choose it for voice chat (Anyone remember TeamSpeak?)