OpenLLM
github.com
OpenLLM
1–10 of 175 posts
Re: OpenLLM
#2Fine-tuning is the most important part, but it is under intense research today, things change fast. I hope they can streamline this process because these smaller models can only compete with big models when they are fine-tuned.
Re: OpenLLM
#3Cool stuff! How does this compare with Fastchat, which seems like another open source project that helps run LLM models?
At a glance, it seems like it's going for lots of similar goals (run LLMs with interoperable APIs):
Re: OpenLLM
#4Fine-tuning is the most important part, but it is under intense research today, things change fast. I hope they can streamline this process because these smaller models can only compete with big models when they are fine-tuned.
Yes but fine tuning requires a lot more gpu memory and is thus much more expensive, complicated and out of reach of most people. To fine tune a >10B model you still need multiple A100 / H100. Let’s hope that changes with quantized fine tuning, forward pass only etc.
Re: OpenLLM
#5Cool stuff! How does this compare with Fastchat, which seems like another open source project that helps run LLM models? At a glance, it seems like it's going for lots of similar goals (run LLMs with interoperable APIs): https://github.com/lm-sys/FastChat
OpenLLM in comparison focuses more on building LLM apps for production. For example, the integration with LangChain + BentoML makes it easy to run multiple LLMs in parallel across multiple GPUs/Nodes, or chain LLMs with other type of AI/ML models, and deploy the entire pipeline on Kubernete (via Yatai or BentoCloud).
Disclaimer: I helped build BentoML and OpenLLM.
Re: OpenLLM
#6Stray thought: It would be better to specify NNs in terms of their training-size to weight-size in bytes. Rather than "No. Parameters", or at least, this ratio with the number of parameters.
So, eg., I'd imagine ChatGPT would be, say: 100s PB in 0.5TB.
The number of parameters is a nearly meaningless metric, consider, eg., that if all the parameters covary then there's one "functional" parameter.
The compression ratio tells you the real reason why a NN performs. Ie., you can have 100s bns of parameters, but without that 100s PB -> 0.5TB, which you can't afford, it's all rather pointless.
Re: OpenLLM
#7Fine-tuning is the most important part, but it is under intense research today, things change fast. I hope they can streamline this process because these smaller models can only compete with big models when they are fine-tuned.
What exactly do you mean here that the smaller models can compete with the the larger once they are fine-tuned? What about once the larger models are fine-tuned? Are they then out of reach of the fine-tuned smaller models?
Re: OpenLLM
#8Fine-tuning is the most important part, but it is under intense research today, things change fast. I hope they can streamline this process because these smaller models can only compete with big models when they are fine-tuned.
Yes but fine tuning requires a lot more gpu memory and is thus much more expensive, complicated and out of reach of most people. To fine tune a >10B model you still need multiple A100 / H100. Let’s hope that changes with quantized fine tuning, forward pass only etc.
The OpenLLM team is actively exploring those techniques for streamlining the fine-tuning process and making it accessible!
Re: OpenLLM
#9Stray thought: It would be better to specify NNs in terms of their training-size to weight-size in bytes. Rather than "No. Parameters", or at least, this ratio with the number of parameters. So, eg., I'd imagine ChatGPT would be, say: 100s PB in 0.5TB. The number of parameters is a nearly meaningless metric, consider, eg., that if all the parameters covary then there's one "functional" parameter. The compression rati…
Well it's not a completely meaningless metric as it immediately tells you roughly how much memory you need to load it, which is kind of important?
Re: OpenLLM
#10Stray thought: It would be better to specify NNs in terms of their training-size to weight-size in bytes. Rather than "No. Parameters", or at least, this ratio with the number of parameters. So, eg., I'd imagine ChatGPT would be, say: 100s PB in 0.5TB. The number of parameters is a nearly meaningless metric, consider, eg., that if all the parameters covary then there's one "functional" parameter. The compression rati…
Well it's not a completely meaningless metric as it immediately tells you roughly how much memory you need to load it, which is kind of important?
If you look at my suggestion, it's to state exactly that memory -- rather than to estimate based on bits/parameter.