MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
1–10 of 58 posts
Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#2Nice, could one use this to train models for Windows PCs also? I don't have a lot of ram.
Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#3> MobileLLM-125M/350M attains a remarkable 2.7%/4.3% accuracy boost over preceding 125M/350M SoTA models on zero-shot commonsense reasoning tasks
Small models, slightly improved, probably still not good enough for the same use as online models. Nothing wrong with incremental progress, however.
1.5B parameter model does seem to be a pretty decent step up, even beating larger models by a wide margin. I'm not sure why they didn't go larger -- having a more efficient model that fits on hardware the size of the RPi could be a gamechanger (IIRC TinyLlama 7B does run, barely).
Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#4While this is interesting, I wonder what the use case is, other than better autocomplete?
Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#5What apps can one currently use to run them on say an iPhone? Only aware of the MLC one which has literally 3 old models only
Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#6While this is interesting, I wonder what the use case is, other than better autocomplete?
It could power simple agents like Siri under the hood. Helping with natural language understanding, intent classification, retrieval, and other agent tasks.
Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#7Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#8Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#9Am I missing something but can't something like distillation help here ?
Re: MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use
#10While this is interesting, I wonder what the use case is, other than better autocomplete?
Local agent like siri that can do simple tasks, and route more complex requests.