Small AI Models Gain Traction In places with unreliable networks
spectrum.ieee.org
Small AI Models Gain Traction In places with unreliable networks
1–10 of 92 posts
Re: Small AI Models Gain Traction In places with unreliable networks
#299% of the model "work" (meaning the connection to your computer) is just spinning a spinner - something that makes me want to wrap it with a mosh shell so I can just keep moving from network to network.
Re: Small AI Models Gain Traction In places with unreliable networks
#3Fascinating to wonder whether the bigger model finds fewer or more counterfeits than the on-device one.
Re: Small AI Models Gain Traction In places with unreliable networks
#4Is anyone making LLM-in-a-box for emergency supply kits yet?
I feel that would be handy in all sorts of situations when networks are down.
Re: Small AI Models Gain Traction In places with unreliable networks
#5Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.
Can you expand what you mean?
Re: Small AI Models Gain Traction In places with unreliable networks
#6Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.
Can you expand what you mean?
They want to ask the iOS Foundation model (frontier on device intelligence for something small) for instance about emergency procedures and life-saving info. I wouldn’t trust that model with much at all though. More likely to find what you need from miniature survival guides.
Re: Small AI Models Gain Traction In places with unreliable networks
#7Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.
This is couched in prepper nonsense, but it's got LLM, WikiPedia, maps, etc. A bunch of genuinely useful stuff to keep on a USB stick or whatever: https://www.projectnomad.us/
But, the current model you really want for an emergency kit is Gemma 4 12B QAT 4-bit. At ~7GB on disk, it's small enough to run on a tablet or any modern computer, slowly if you don't have a GPU or modern Apple silicon, but exceedingly smart for its size, excellent vision capabilities, good tool user, surprisingly good reasoning.
Re: Small AI Models Gain Traction In places with unreliable networks
#8Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.
> LLM-in-a-box for emergency
For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.)
In contrast, imagine spending the money to build a beefy LLM-running computer with good GPU/RAM, and somehow mothballing it (to depreciate, unused) in a "safe" location for the big earthquake/flood/etc... Then when the disaster strikes and you dig it out, how will you power it when you need it, and for long enough to do anything useful?
Even if wall-current civilization is 20 miles away on the other side of the mountain, are you going to carry it on your back, or are you going to carry food and water to live? If you do drag it there, are they going to let you run it when it cuts into light for surgery or heat to sterilize drinking water?
Re: Small AI Models Gain Traction In places with unreliable networks
#9Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.
> LLM-in-a-box for emergency For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.) In contrast, imagine…
You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?
Re: Small AI Models Gain Traction In places with unreliable networks
#10I think neuro-symbolic AI has a lot of potential here, since small models can handle a lot of conversational inputs, while relying on wired-in solvers for more complex symbolic math/computation needs. https://jjd.io/posts/swollm-bbh-leaderboard.html