Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small AI Models Gain Traction In places with unreliable networks

spectrum.ieee.org

Post reply on HN
Print view View original

41–50 of 92 posts

Re: Small AI Models Gain Traction In places with unreliable networks

#41

Post by swiftcoder »

zmgsabst wrote: post #30
Terr_ wrote: post #26

Earlier quoted context omitted.

I feel this is going into increasingly-unlikely mixes of constraints and needs in order to try to keep a "wouldn't it be cool if" hypothetical-tool dream alive. [0] But OK, let's assume that: The power is out, but you have a generator with so much fuel you can run a desktop just fine; Your neighborhood will somehow make a mesh network; Your neighbors need some already stored information and the best solution for that…

Option 1 sounds better: I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me. - - - More broadly… You call it unlikely mixes, but we see it all the time: - people already have a computer for gaming or work - people (ie, “preppers” like we’re discussing) buy a generator for emergencies - local emergency response sets up mesh networking during disasters,…

> I’m only out the cost of the drive, which is like $40 and doesn’t require anybody on the other side cooperate with me.

At current prices you are also out about $4k for a Spark to actually run the inference on, if you want a full LLM in a low-power package.

In general, I'm not sure why one would want to pin your survival to an expensive, hallucination-prone data source, when an offline copy of wikipedia with a little vector search attached to a Raspberry Pi can fulfil the same role...

Re: Small AI Models Gain Traction In places with unreliable networks

#42

Post by simianwords »

fasterik wrote: post #40
simianwords wrote: post #34

Earlier quoted context omitted.

No this will never work. Domain specific models will never be a thing because intelligence carries over and compounds. Why didn’t OpenAI release a math specific model? Why not a literature specific one? Why do they instead have generic models of different sizes? And how did all labs converge on this? Why does Fable just not train on non cybersec and non biology data but instead have clearly costly and annoying classi…

Your examples (math, literature) involve natural language. It stands to reason that a general language model will be more competitive in those domains. If you want examples of successful domain-specific models, look at AlphaZero and AlphaFold. LLMs aren't anywhere close to achieving that level of competence at abstract strategy games or protein folding. "This will never work" is a pretty confident assertion for a fie…

> I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet).

This is what the parent said. AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine. This is the industry consensus.

Re: Small AI Models Gain Traction In places with unreliable networks

#43

Post by fasterik »

simianwords wrote: post #42
fasterik wrote: post #40

Earlier quoted context omitted.

Your examples (math, literature) involve natural language. It stands to reason that a general language model will be more competitive in those domains. If you want examples of successful domain-specific models, look at AlphaZero and AlphaFold. LLMs aren't anywhere close to achieving that level of competence at abstract strategy games or protein folding. "This will never work" is a pretty confident assertion for a fie…

> I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). This is what the parent said. AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine. This is the industry consensus.

>AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine.

That's a straw man. Nobody thinks AGI will rise out of domain-specific systems. The question is whether domain-specific systems are necessary for AGI.

Of course, the problem is that AGI isn't a well-defined concept. But if we define it as achieving superhuman performance across several hundred domains where there are objective measures of success, it doesn't seem far-fetched to predict that it will involve some general reasoning system paired with a bunch of specialized modules.

Re: Small AI Models Gain Traction In places with unreliable networks

#44

Post by simianwords »

fasterik wrote: post #43
simianwords wrote: post #42

Earlier quoted context omitted.

> I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). This is what the parent said. AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine. This is the industry consensus.

>AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine. That's a straw man. Nobody thinks AGI will rise out of domain-specific systems. The question is whether domain-specific systems are necessary for AGI. Of course, the problem is that AGI isn't a well-defined concept. But if we define it as achieving superhuman performance across several hundred domains where ther…

The parent said

> I don't foresee AGI arising out training bigger LLMs

I agree that AGI will involve tool usage but not only involving domain specific AI models.

But lets try to find the discriminating point in the discussion - do you believe AGI will necessarily involve training bigger LLM's or not?

I believe they are necessary. WBU?

Re: Small AI Models Gain Traction In places with unreliable networks

#45

Post by fasterik »

simianwords wrote: post #44
fasterik wrote: post #43

Earlier quoted context omitted.

>AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine. That's a straw man. Nobody thinks AGI will rise out of domain-specific systems. The question is whether domain-specific systems are necessary for AGI. Of course, the problem is that AGI isn't a well-defined concept. But if we define it as achieving superhuman performance across several hundred domains where ther…

The parent said > I don't foresee AGI arising out training bigger LLMs I agree that AGI will involve tool usage but not only involving domain specific AI models. But lets try to find the discriminating point in the discussion - do you believe AGI will necessarily involve training bigger LLM's or not? I believe they are necessary. WBU?

You're still intentionally misreading the OP's statement. If you read it again in context, they're clearly saying that they think training bigger LLMs is not sufficient. I think I agree with that statement, but my confidence is pretty low.

No, I don't think LLMs are necessary for AGI at all. I think there are multiple paths to AGI, some of which involve LLMs and some which don't.

Re: Small AI Models Gain Traction In places with unreliable networks

#46

Post by [deleted] »

plastic-enjoyer wrote: post #38
andy99 wrote: post #33

Earlier quoted context omitted.

General purpose models are always more robust and generally better than smaller narrower models. My bet is that compute will catch up and any “small” model will still be generally capable, just smaller than sota, rather than intentionally narrow. The exception would be for very well defined tasks where the data distribution never varies, but these are rare and don’t really need “AI” anyway when they do exist.

> General purpose models are always more robust and generally better than smaller narrower models. What do you mean with more robust?

[deleted]

Re: Small AI Models Gain Traction In places with unreliable networks

#47

Post by crsx_ »

iamflimflam1 wrote: post #29
skybrian wrote: post #9

Earlier quoted context omitted.

You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?

You may benefit from an embedding approach for semantic search. Not sure what an LLM would give you on top of that.

If the emergency is in a foreign country, being able to communicate with locals would likely be a benefit - and a domain specific trained model could translate better than general purpose translators.

In general, I think speech as input/output is under-explored. In the emergency scenario, in a stressful environment, having an expert in your ear you can talk to should work much better than having a big manual book to look up specific cases.

Re: Small AI Models Gain Traction In places with unreliable networks

#49

Post by visarga »

N_Lens wrote: post #18

I strongly believe this premise in the article is correct - we will see a lot of tiny, hyper specialized models for individual tasks, and perhaps that will converge with an orchestration layer for a generalized intelligence that controls these specialized tiny models, that will be quite capable. I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet). It's actually h…

I think the harness and local context should supply that missing piece between general model and bespoke application. Each application has its own context and action quirks that don't generalize well. Maybe it's just 5% but that is genuinely specific. So its rightful place is in context engineering.

I have a long-ass post about how this could be implemented. https://old.reddit.com/r/VisargaPersonal/comments/1um9uyv/st...

Re: Small AI Models Gain Traction In places with unreliable networks

#50

Post by Lio »

iamflimflam1 wrote: post #29
skybrian wrote: post #9

Earlier quoted context omitted.

You will probably want a search engine though. Perhaps a small LLM would work well as a component for that?

You may benefit from an embedding approach for semantic search. Not sure what an LLM would give you on top of that.

I think an LLM would give a "conversational" experience to search.

That's handy for situations where you might not really understand what you need to search for. Any search system that can ask you clarifying questions is going to be a big improvement.

Or where you need to combine several steps together but you don't yet know what those steps are.

There's probably other technologies that could do that, requiring lower resources but they'll come with different trade-offs around configuration.

Just having a Raspberry PI, a offline copy wikipedia and a RAG enabled small LLM would be quite useful or at least entertaining if you have to go off grid.

Post reply on HN