Can't wait to be killed by my toaster because some sexy mossad agent seduced it.
Have you ever tried to indulge an all-consuming urge to kill when you don't have opposable thumbs? Or hands? Or anything other than a bread slot?
Small AI Models Gain Traction In places with unreliable networks
51–60 of 92 posts
Re: Small AI Models Gain Traction In places with unreliable networks
#52Earlier quoted context omitted.
> LLM-in-a-box for emergency For most actual emergency scenarios, a device that focuses on storage of large amounts of prepared normal reference material [0] will be wayyyyy cheaper, more durable, portable, and able to run on batteries or being constantly plugged into a somehow-still-normal electrical grid. (Think an e-ink tablet that can run off a 5V battery pack buffering a literal handcrank.) In contrast, imagine…
You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?
> You will probably want a search engine though.
The search engine is indeed the last missing component from a sovereign stack. But I think this could be solved locally with little cost. Instead of indexing content on the web we should be indexing sources themselves - where to look for X? - like forums, blogs, docs, feeds, and specialized search engines. We could collectively amass millions of these search stubs that can be used by local models to go and fetch fresh information from the source directly. This means separating the routing layer from the information layer, we don't need to keep information cached from the whole internet locally. The search stubs could fit in a few GB about same size with the local LLM. The cool thing is that sources change much slower than information itself, so the search stub database could be refreshed at a slower pace. We could combine a few million generic stubs with a few hundred personal stubs generated from our own activities. It is trivial to generate these stubs by piggy backing on frontier models.
Re: Small AI Models Gain Traction In places with unreliable networks
#53Re: Small AI Models Gain Traction In places with unreliable networks
#54I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable. I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). It's actually h…
I think future is probably more similar to speculative execution (inference/decoding). A small LLM is used to speculate and a large LLM is used to confirm if needed. If the small LLM is accurate enough on N tokens it’s cheap for the large LLM to say looks good and keep moving along.
Re: Small AI Models Gain Traction In places with unreliable networks
#55Earlier quoted context omitted.
I've been mulling over a good use of a large philanthropy spend in the next decade, and I would love to build a bunch of hardware "oracles" that include an LLM. Ideally solid state, visual/audio, solar + usb-c, so, good in a lot of doomsday scenarios as well as just out hiking. It's a fun thought experiment. I imagine making like 1 million of them, they could be sold and genuinely useful, but also given away; once ow…
I feel like you could get a lot more quality of life improvement for more people with the money if you spent it on low tech solutions, eg more efficient cooking stoves for people still cooking with biomass, or solar microgrids for areas without electricity.
No doubt there are a hundred more direct things one could do. On the other hand this would include instructions for all those things and could advise on the building! I think it would be a nice thing to have in the world.
Re: Small AI Models Gain Traction In places with unreliable networks
#56Where is a good place to start with training SLM these days if you don't have the compute locally?
I rent cheap GPU based instances by the hour and run on those. Nothing fancy, but $20 can get you a decent amount of compute on a A6000 or H100.
Re: Small AI Models Gain Traction In places with unreliable networks
#57[flagged]
Re: Small AI Models Gain Traction In places with unreliable networks
#58I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable. I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). It's actually h…
[flagged]
Re: Small AI Models Gain Traction In places with unreliable networks
#59Can't wait to be killed by my toaster because some sexy mossad agent seduced it.
This made me laugh out loud :)
Re: Small AI Models Gain Traction In places with unreliable networks
#60I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable. I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). It's actually h…
I think future is probably more similar to speculative execution (inference/decoding). A small LLM is used to speculate and a large LLM is used to confirm if needed. If the small LLM is accurate enough on N tokens it’s cheap for the large LLM to say looks good and keep moving along.
[dead]