[flagged]
Small AI Models Gain Traction In places with unreliable networks
71–80 of 92 posts
Re: Small AI Models Gain Traction In places with unreliable networks
#72Earlier quoted context omitted.
General purpose models are always more robust and generally better than smaller narrower models. My bet is that compute will catch up and any “small” model will still be generally capable, just smaller than sota, rather than intentionally narrow. The exception would be for very well defined tasks where the data distribution never varies, but these are rare and don’t really need “AI” anyway when they do exist.
> General purpose models are always more robust and generally better than smaller narrower models. What do you mean with more robust?
Less weird unexpected failures, more innate ability to handle edge cases gracefully. Quite important when you're running high on automation and low on oversight.
Re: Small AI Models Gain Traction In places with unreliable networks
#73I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable. I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). It's actually h…
General purpose models are always more robust and generally better than smaller narrower models. My bet is that compute will catch up and any “small” model will still be generally capable, just smaller than sota, rather than intentionally narrow. The exception would be for very well defined tasks where the data distribution never varies, but these are rare and don’t really need “AI” anyway when they do exist.
You're getting downvoted, but you're completely right. There are very few cases in which narrowing a model down is buying you anything worthwhile.
It seems like for LLMs, "general intelligence" is expensive, but "one more domain" is fairly cheap.
Re: Small AI Models Gain Traction In places with unreliable networks
#74Earlier quoted context omitted.
If you are rebuilding society most of this knowledge is useless for centuries. You don't have enough labor to build and maintain factories. You will spend centuries in the hunter gather phase struggling to survive, while slowly building agriculture. You will be lucky if you can teach your grandkids to read - since that will be a useless skill. Print important knowledge on paper and store it in a desert. in 2000 years…
There is no rebuilding society because of energy. All of the oil and coal that could be extracted by a civilization that only has wood for fuel has been used. If we go back to the Stone Age, we aren’t returning to the present. If we’re lucky, we will get to the Middle Ages.
There is still a lot of coal and oil to be had. It isn't as easy as it used to be, but there is still a lot. If we get to the middle ages then society can build a lot with knowledge. More efficient iron production than what the middle ages (when still using wood) had for example would be useful, and our knowledge of technology may allow bicycles sooner (bicycles need higher technology than a steam engine)
Re: Small AI Models Gain Traction In places with unreliable networks
#75Earlier quoted context omitted.
You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?
> You will probably want a search engine though. The search engine is indeed the last missing component from a sovereign stack. But I think this could be solved locally with little cost. Instead of indexing content on the web we should be indexing sources themselves - where to look for X? - like forums, blogs, docs, feeds, and specialized search engines. We could collectively amass millions of these search stubs that…
[deleted]
Re: Small AI Models Gain Traction In places with unreliable networks
#76Earlier quoted context omitted.
I feel this is going into increasingly-unlikely mixes of constraints and needs in order to try to keep a "wouldn't it be cool if" hypothetical-tool dream alive. [0] But OK, let's assume that: The power is out, but you have a generator with so much fuel you can run a desktop just fine; Your neighborhood will somehow make a mesh network; Your neighbors need some already stored information and the best solution for that…
Option 1 sounds better: I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me. - - - More broadly… You call it unlikely mixes, but we see it all the time: - people already have a computer for gaming or work - people (ie, “preppers” like we’re discussing) buy a generator for emergencies - local emergency response sets up mesh networking during disasters,…
> Have you ever tried to use a handbook you’re not intimately familiar with during an emergency? It’s rough.
Sounds like the absolute worst time to rely on a crappy little model that will inevitably hallucinate.
Re: Small AI Models Gain Traction In places with unreliable networks
#77[dead]
Re: Small AI Models Gain Traction In places with unreliable networks
#78Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.
I don't if anyone is doing this yet, but I think a small LLM w/ data sets for RAG reachable via APRS and LoRa would be very useful, not just as an individual but for the community around you.
Re: Small AI Models Gain Traction In places with unreliable networks
#79Is anyone making LLM-in-a-box for emergency supply kits yet? I feel that would be handy in all sorts of situations when networks are down.
Oh my... I can think of a 101 things more useful in actual emergencies than an LLM-in-a-box. Unless you have a weird definition of "emergency" (if so: please define).
Re: Small AI Models Gain Traction In places with unreliable networks
#80Earlier quoted context omitted.
They want to ask the iOS Foundation model (frontier on device intelligence for something small) for instance about emergency procedures and life-saving info. I wouldn’t trust that model with much at all though. More likely to find what you need from miniature survival guides.
> They want to ask the iOS Foundation model (frontier on device intelligence for something small) This is a bit of a straw man, TBH. For one thing, "LLM-in-a-box" doesn't necesssarily imply a device as small as a phone. For another, you'd need to convince people that the iOS Foundation model is the "frontier" of LLMs that run on phones when it is really not. AFAIK it is noticeably outperformed by the Gemma 4 E2B mode…
I meant the new 20B param Foundation models, not the ones from last year. But sure Gemma might be better