[flagged]
Small AI Models Gain Traction In places with unreliable networks
21–30 of 92 posts
Re: Small AI Models Gain Traction In places with unreliable networks
#22I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable. I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). It's actually h…
What about recent models providing correct proofs to open math problems?
Re: Small AI Models Gain Traction In places with unreliable networks
#23Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.
electricity outage and battery running out is the end game for any real prolonged external emergency. Internet connection is just the soft edge.
Re: Small AI Models Gain Traction In places with unreliable networks
#24I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable. I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). It's actually h…
What about recent models providing correct proofs to open math problems?
What about it?
Re: Small AI Models Gain Traction In places with unreliable networks
#25SLMs for the rescue!
Re: Small AI Models Gain Traction In places with unreliable networks
#26Earlier quoted context omitted.
> LLM-in-a-box for emergency For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.) In contrast, imagine…
You can run a small model off a home generator — so in an emergency, you’d turn on both the generator and information service, eg, a mesh for “quick” responses querying that huge collection of information. That way your machine that, eg, normally plays video games or does AI work can support relief efforts by supporting emergency response IT. You don’t need to mothball the machine, just have an “emergency” boot USB t…
I feel this is going into increasingly-unlikely mixes of constraints and needs in order to try to keep a "wouldn't it be cool if" hypothetical-tool dream alive. [0]
But OK, let's assume that: The power is out, but you have a generator with so much fuel you can run a desktop just fine; Your neighborhood will somehow make a mesh network; Your neighbors need some already stored information and the best solution for that is texting a chatbot rather than a survival/emergency handbook or Wikipedia; Your mesh-network will also be good enough to match the time-sensitivity of the questions.
Under those assumption, which of these sounds better?
1. Buying an "LLM-in-a-box for emergency supply kits", which you deploy so that your neighbors can ask questions (text over the mesh) of the offline chatbot.
2. Buying a satellite internet transciever for your emergency supply kit, so that your neighbors can ask questions of a much better chatbot and communicate with human experts, their worried relatives, and coordinate with rescue/relief efforts...
Re: Small AI Models Gain Traction In places with unreliable networks
#27Where is a good place to start with training SLM these days if you don't have the compute locally?
Re: Small AI Models Gain Traction In places with unreliable networks
#28I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable. I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). It's actually h…
What about recent models providing correct proofs to open math problems?
I haven't tried it, but I saw Leanstral, an LLM specialized in writing Lean proofs, posted on HN recently and it claims to outperform some larger general purpose models. It didn't beat Claude Opus, but it seems to do decently at one tenth the cost. It's plausible that further research could yield other models that are smaller and more effective at limited tasks, reversing the trend of ever growing models.
Re: Small AI Models Gain Traction In places with unreliable networks
#29Earlier quoted context omitted.
> LLM-in-a-box for emergency For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.) In contrast, imagine…
You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?
You may benefit from an embedding approach for semantic search. Not sure what an LLM would give you on top of that.
Re: Small AI Models Gain Traction In places with unreliable networks
#30Earlier quoted context omitted.
You can run a small model off a home generator — so in an emergency, you’d turn on both the generator and information service, eg, a mesh for “quick” responses querying that huge collection of information. That way your machine that, eg, normally plays video games or does AI work can support relief efforts by supporting emergency response IT. You don’t need to mothball the machine, just have an “emergency” boot USB t…
I feel this is going into increasingly-unlikely mixes of constraints and needs in order to try to keep a "wouldn't it be cool if" hypothetical-tool dream alive. [0] But OK, let's assume that: The power is out, but you have a generator with so much fuel you can run a desktop just fine; Your neighborhood will somehow make a mesh network; Your neighbors need some already stored information and the best solution for that…
Option 1 sounds better:
I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me.
- - -
More broadly…
You call it unlikely mixes, but we see it all the time:
- people already have a computer for gaming or work
- people (ie, “preppers” like we’re discussing) buy a generator for emergencies
- local emergency response sets up mesh networking during disasters, both official and unofficial
Have you ever tried to use a handbook you’re not intimately familiar with during an emergency? It’s rough.
For personal preparedness, nothing replaces familiarity and practice — eg, weekend survival trips and reading your manual ahead of time.
But for providing information in a random lookup manner to unpracticed people who weren’t prepared? Yes, I think an LLM/chatbot is the practical way to operationalize all that information which you stored (eg, survival guides or machine manuals).
Also, it’s unlikely a general purpose chatbot would be superior at survival advice to one specialized for that purpose — and indeed, is likely to refuse your questions as “unsafe” or “criminal”.