141–150 of 213 posts
What infrastructure does it require to test?
Ugh. I just spent the past few days reading this exact paper and preparing a detailed presentation on it for my job as an AI researcher, and headlines like this make me roll my eyes very hard. MEGABYTE is indeed a cool new architecture, but it is still very much just a proof of concept at the moment. The paper shows that the model can compete with (but not decimate) vanilla Transformers on the scale of ~1B parameters…
Earlier quoted context omitted.
You mean the average brain with about 100 billion neurons with about 1000 connections each bringing it to around 100 trillion connections. With an estimated 1000 "AI" neurons required per biologial neuron. I don't think you are givin these "below average" intelligence individuals enough credit. What we consider a genius is the equivalent of a dog show obstacle course. We measure intelligence/genius as whatever is har…
Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc. Majority of human brain computation is spent on things that are simply not necessary for computer models (how to wiggle limbs, mouth, eyes etc). Current LLM are impressive, but we know they can be much more efficient - we're using very low quality training data, we don…
That's not to say we can't do this with computers and less computational power. However, it's really improbable that a couple layers of adaptability on an artificial neuron network will be anywhere near sufficient to simulate intelligence in even a rodent.
Earlier quoted context omitted.
> Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc. Wheels are trivial but useless without bearings. Bearings most certainly aren't trivial. All the things you listed further are dependent on bearings somewhere.
AFAIK both exist in biology. Bacterial flagellum is effectively a motor, and has a working wheel-like structure. IIRC, some crickets had an equivalent of a bearing somewhere in their anatomy too. Evolution is a greedy, lazy optimizer, so it promotes things that work a-ok for a given environment. It's also worth noting that wheels alone are not too useful for transportation, as they're only half of the picture. The ot…
https://www.livescience.com/39577-insects-with-leg-gears-dis...
Earlier quoted context omitted.
Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc. Majority of human brain computation is spent on things that are simply not necessary for computer models (how to wiggle limbs, mouth, eyes etc). Current LLM are impressive, but we know they can be much more efficient - we're using very low quality training data, we don…
> Wheels are trivial, however not present in biology. Because roads don't exist in nature..... imagine trying to out run a predator if you just had wheels but no roads. > Nature creates tentacles, not jet engines, nuclear energy etc Under water I think you'll find there is jet propulsion. Plants and animals are powered by nuclear energy - the remote fusion reaction is in the sky. Why have an internal nuclear reactor…
Technically, yes, but most people are going to be thinking of the spinning turbines of doom with the absurdly hot fire in the middle, and that isn't.
Earlier quoted context omitted.
AFAIK both exist in biology. Bacterial flagellum is effectively a motor, and has a working wheel-like structure. IIRC, some crickets had an equivalent of a bearing somewhere in their anatomy too. Evolution is a greedy, lazy optimizer, so it promotes things that work a-ok for a given environment. It's also worth noting that wheels alone are not too useful for transportation, as they're only half of the picture. The ot…
If that's the case why are we sending wheeled rovers to mars/moon and not something with legs? In any case this discussion is going sideways, the point is that nature doesn't have monopoly on being optimal. This also applies to intelligence/learning/modeling something better than brain.
Interestingly the latest Mars rover also includes a small helicopter, another technology which requires spinning something on a bearing and does not commonly exist in nature.
Earlier quoted context omitted.
> it's not a matter of having a 100x more powerful LLM, I think we all can agree that even the best LLM currently is not AGI. That's not what being disputed here I think. However a 100x more powerful LLM is not just 100x better at recall. A 100x more powerful LLM is not just 100x better at being stupid hallucinatory parrot. A model that is just 100x bigger is not necessarily 100x more powerful if you define power is…
Define grounding things in reality. We only have our 5 senses to go off of. Meta has already put out one multimodal model incorporating multiple data types, openai is undoubtedly working on it too.
Balance, proprioception, hunger, …
Ugh. I just spent the past few days reading this exact paper and preparing a detailed presentation on it for my job as an AI researcher, and headlines like this make me roll my eyes very hard. MEGABYTE is indeed a cool new architecture, but it is still very much just a proof of concept at the moment. The paper shows that the model can compete with (but not decimate) vanilla Transformers on the scale of ~1B parameters…
Earlier quoted context omitted.
Nobody said that nature is optimal. Wheels are trivial, however not present in biology. Nature creates tentacles, not jet engines, nuclear energy etc. Majority of human brain computation is spent on things that are simply not necessary for computer models (how to wiggle limbs, mouth, eyes etc). Current LLM are impressive, but we know they can be much more efficient - we're using very low quality training data, we don…
The lack of efficiency in the brain is a trade off for adaptability. There's no doubt that we can transmit signals faster than neurons can fire but the cost is drastically reduced adaptation. There are micro, meso, and macroscopic networks in the brain that have different degrees of "adaptability". This isn't even considering the variety in neurons or the additional signalling cascades by non neuronal tissue. How doe…