Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small AI Models Gain Traction In places with unreliable networks

spectrum.ieee.org

Post reply on HN
Print view View original

11–20 of 92 posts

Re: Small AI Models Gain Traction In places with unreliable networks

#11

Post by vessenes »

tim-fan wrote: post #4

Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.

I've been mulling over a good use of a large philanthropy spend in the next decade, and I would love to build a bunch of hardware "oracles" that include an LLM. Ideally solid state, visual/audio, solar + usb-c, so, good in a lot of doomsday scenarios as well as just out hiking. It's a fun thought experiment. I imagine making like 1 million of them, they could be sold and genuinely useful, but also given away; once owned, you could use them, or store and put in an emergency box, bury next to the 10k year clock.. a lot of possibilities.

Re: Small AI Models Gain Traction In places with unreliable networks

#13

Post by weikju »

Terr_ wrote: post #8
tim-fan wrote: post #4

Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.

> LLM-in-a-box for emergency For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.) In contrast, imagine…

> If you do carry it to an enclave of civilization that has the right power, are they going to let you run it when it cuts into light for surgeons or heat to sterilize water?

Knowing humans? They'd probably take it by force and run it for themselves instead of providing light and heat to surgeons and water sterilizers...

/daily dose of cynism

Re: Small AI Models Gain Traction In places with unreliable networks

#14

Post by zmgsabst »

Terr_ wrote: post #8
tim-fan wrote: post #4

Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.

> LLM-in-a-box for emergency For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.) In contrast, imagine…

You can run a small model off a home generator — so in an emergency, you’d turn on both the generator and information service, eg, a mesh for “quick” responses querying that huge collection of information.

That way your machine that, eg, normally plays video games or does AI work can support relief efforts by supporting emergency response IT. You don’t need to mothball the machine, just have an “emergency” boot USB than can run the services from your home generator.

You don’t even need to bring it with you: turn it on and leave it “best effort” at home, while you continue to use it via WAN.

Re: Small AI Models Gain Traction In places with unreliable networks

#15

Post by adrianN »

vessenes wrote: post #11
tim-fan wrote: post #4

Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.

I've been mulling over a good use of a large philanthropy spend in the next decade, and I would love to build a bunch of hardware "oracles" that include an LLM. Ideally solid state, visual/audio, solar + usb-c, so, good in a lot of doomsday scenarios as well as just out hiking. It's a fun thought experiment. I imagine making like 1 million of them, they could be sold and genuinely useful, but also given away; once ow…

I feel like you could get a lot more quality of life improvement for more people with the money if you spent it on low tech solutions, eg more efficient cooking stoves for people still cooking with biomass, or solar microgrids for areas without electricity.

Re: Small AI Models Gain Traction In places with unreliable networks

#17

Post by rasz »

tim-fan wrote: post #4

Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.

Smallest local model able to work with offline wikipedia dump would be one step above just having an offline wikipedia dump.

Re: Small AI Models Gain Traction In places with unreliable networks

#18

Post by N_Lens »

I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable.

I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet).

It's actually how organic brains work - specialized tasks are offloaded to local cortical columns. The overall coordination between these sub-brains creates emergent skills/abilities.

Re: Small AI Models Gain Traction In places with unreliable networks

#20

Post by rtpg »

skybrian wrote: post #9
Terr_ wrote: post #8

Earlier quoted context omitted.

> LLM-in-a-box for emergency For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.) In contrast, imagine…

You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?

grep works well!
Post reply on HN