Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Natural language benchmarks don’t measure AI models’ general knowledge well

venturebeat.com

Post reply on HN
Print view View original

41–50 of 71 posts

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#41

Post by tsimionescu »

mjburgess wrote: post #38
IfOnlyYouKnew wrote: post #22

Earlier quoted context omitted.

It amazes me people think their brain works differently. It's pretty close to believing in body/mind dualism, the only thing in neuroscience more outdated than Freud. Your brain works within the same laws of physics as the outside world. We don't know how the brain works exactly. But once we understand it, it is unlikely to be qualitatively different from a neural network. On the other side of the equation, the emerg…

> it is unlikely to be qualitatively different from a neural network I'm sorry but this shows a profound misunderstanding of what a NN is, and what the brain is. There are no "neural network"s. The NN algorithm is a method for optimizing the parameters of a piece-wise linear regression model. These regression models have no homology to any brain structure and the process of producing them ("training") has no neurolog…

Very good points! I would add that we don't yet have a complete model of the computational properties of even a non-neural cell, even though we know that all cells exhibit simplistic perception and decision-making.

This is not to say that we should expect some non-turing weirdness or anything 'mystical' like that, but that we just haven't taken the time to analyze the exact mechanisms and to model cells at this level well enough to be able to say that they are similar to any particular computational structure. Thus, to claim that the brain in its entirety is similar to any particular computational structure such as an NN is either wrong or trivial - the only level at which 'similarity' can be claimed is that they are both useful in computation, and likely turing complete. Anything more specific is fanciful thinking.

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#42

Post by tsimionescu »

esc_colon_q wrote: post #8
tsimionescu wrote: post #3

I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world, if only we increase the number of model parameters by a few more orders of magnitude (GPT-3 and the likely strategy for GPT-4). Of course, all this without any model of how likely it is that the knowledge is embedded in the text, such as trying to t…

It is very true that GPT-XYZ is fundamentally limited, like all transformer models, but don't misattribute the root cause to the fact that it's just processing text. The real limitation is that these are feedforward networks that just do perception without any processing of what they've perceived. You can try to hide that fact for a while by increasing the depth of the perception network, basically hard-coding some p…

But the fact that it 'lives' in a world of text is much more fundamental. There is simply not enough information in text to draw a working model of the world. We presuppose such a model in our words and in our communication, so fundamentally any algorithm that is solely trained on text can't learn a model of the world from it.

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#43

Post by maximedb »

ninjin wrote: post #13

Second author here with a link to the arXiv paper: https://arxiv.org/abs/2008.02637 Although I have to say, VentureBeat did much better than most media outlets I have seen writing about current research and what they write is not only accurate but also largely devoid of hype. Perhaps we actually managed to “keep the hype down” as we intended when writing this piece? I will check in on this post now and then if you ha…

Did you also try with "retrieval" models models such as RAG, REALM, etc ?

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#44

Post by i-am-curious »

mjburgess wrote: post #38
IfOnlyYouKnew wrote: post #22

Earlier quoted context omitted.

It amazes me people think their brain works differently. It's pretty close to believing in body/mind dualism, the only thing in neuroscience more outdated than Freud. Your brain works within the same laws of physics as the outside world. We don't know how the brain works exactly. But once we understand it, it is unlikely to be qualitatively different from a neural network. On the other side of the equation, the emerg…

> it is unlikely to be qualitatively different from a neural network I'm sorry but this shows a profound misunderstanding of what a NN is, and what the brain is. There are no "neural network"s. The NN algorithm is a method for optimizing the parameters of a piece-wise linear regression model. These regression models have no homology to any brain structure and the process of producing them ("training") has no neurolog…

It's amazing just how many people believe that neural networks have a close resemblance to the human brain.

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#45

Post by i-am-curious »

LdSGSgvupDV wrote: post #17
tsimionescu wrote: post #3

I'm always amazed that some people think performing statistical analysis (training neural networks) on text can lead to actual intelligence and knowledge about the world, if only we increase the number of model parameters by a few more orders of magnitude (GPT-3 and the likely strategy for GPT-4). Of course, all this without any model of how likely it is that the knowledge is embedded in the text, such as trying to t…

It seems they are on the right direction. But, the number of parameters hit the physical barrier of silicon-based device. To make the neuron network be a strong AI, we may need the parameters in scales of 10^19~10^44 (uncertainty limit and Planck time) estimated from how many frames per second in real world. I personally got a feeling that this is bound to how nature calculate the physical phenomenon. For simulation,…

> To make the neuron network be a strong AI, we may need the parameters in scales of 10^19~10^44 (uncertainty limit and Planck time) estimated from how many frames per second in real world.

There is absolutely no basis for claiming this. It might turn out that there are better algorithms for AI than neural nets. Do you even realize just how many parameters even 10^12 is???

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#46

Post by plutonorm »

mjburgess wrote: post #38
IfOnlyYouKnew wrote: post #22

Earlier quoted context omitted.

It amazes me people think their brain works differently. It's pretty close to believing in body/mind dualism, the only thing in neuroscience more outdated than Freud. Your brain works within the same laws of physics as the outside world. We don't know how the brain works exactly. But once we understand it, it is unlikely to be qualitatively different from a neural network. On the other side of the equation, the emerg…

> it is unlikely to be qualitatively different from a neural network I'm sorry but this shows a profound misunderstanding of what a NN is, and what the brain is. There are no "neural network"s. The NN algorithm is a method for optimizing the parameters of a piece-wise linear regression model. These regression models have no homology to any brain structure and the process of producing them ("training") has no neurolog…

Causal. Imagine a video. Each frame is plotted in space, so instead of a sequence of 2d images, you have a cube. The z axis of the cube shows each frame of the video in turn. This is an equivalent representation to the representation that we are used to. We have just switched the time dimension for a physical dimension. What you are reverently calling causality, is only correlation along the z spacial dimension. In the same way an image can be compressed due to the similarities across it's 2 dimensions, so can a 3d representation of video be compressed along all 3 of it's axis. An appreciation of causality is only induction and induction is only a correlation, if a then probably b. Finding correlations in one of the dimensions you are more familiar with, like the pixels of an image is no different to finding it in time. Within language the elements of time are encoded, just as they are within the video represented as a cube. "The boy threw the ball, the ball landed and then rolled." Just because the time dimension is represented to the neural network within the input parameters of a single iteration of the neural network, does not make it any less able to understand correlations across time.

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#47

Post by s_brady »

mjburgess wrote: post #36
Isinlor wrote: post #5

Earlier quoted context omitted.

Understanding of causality is very likely an emergent property. While extremely important, it's unlikely that we have some hard-coded low level architecture of causal inference in brains. It probably will just arise as a necessity of grounded understanding of the world.

I'm not exactly sure what you mean. The conditions for causal inference being possible are pretty clear and have to do with the intentional modification of the local environment. The intention to achieve some new environmental state, and your action to bring it about, is a dynamical activity that enables "deep" model building. Causal inference is not going to be some "module" of the brain... it requires a body. When…

Your comment is interesting but seems to conflate two separate though related areas. The need for a body arises from the embodied cognition school of AI which suggests that intelligence is fundamentally embodied, hence the need for a robot equivalent to a body for truly understanding language.

However this does not necessarily have to be related to causality, and counterfactual statements about a causal model. The math behind counterfactuals and causality is actually well understood now (see any of Pearl's books). It does not actually require that a system be embodied, just that the system have some suitable (and correct) causal model of the world.

It would of course be amazing to have both in one system, but that is not required. An AI system that understood causality and language could be bootstrapped from causal models supplied by humans - or even other AIs :)

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#48

Post by amcoastal »

mjburgess wrote: post #36
Isinlor wrote: post #5

Earlier quoted context omitted.

Understanding of causality is very likely an emergent property. While extremely important, it's unlikely that we have some hard-coded low level architecture of causal inference in brains. It probably will just arise as a necessity of grounded understanding of the world.

I'm not exactly sure what you mean. The conditions for causal inference being possible are pretty clear and have to do with the intentional modification of the local environment. The intention to achieve some new environmental state, and your action to bring it about, is a dynamical activity that enables "deep" model building. Causal inference is not going to be some "module" of the brain... it requires a body. When…

I disagree. There are other ways to observe and learn cause and effect than by physical interaction. We learn new causes and effect of many things each day by reading or watching. There are also most likely examples of humans who never had control over their physical world (through disability or whatnot) that came to understand cause and effect.

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#49

Post by patrick-lewis »

maximedb wrote: post #43
ninjin wrote: post #13

Second author here with a link to the arXiv paper: https://arxiv.org/abs/2008.02637 Although I have to say, VentureBeat did much better than most media outlets I have seen writing about current research and what they write is not only accurate but also largely devoid of hype. Perhaps we actually managed to “keep the hype down” as we intended when writing this piece? I will check in on this post now and then if you ha…

Did you also try with "retrieval" models models such as RAG, REALM, etc ?

Hi, first author here. Yes, we evaluated 3 retrieval-based models, DPR, RAG and FID - check out the paper for the numbers (https://arxiv.org/pdf/2008.02637.pdf)

Re: Natural language benchmarks don’t measure AI models’ general knowledge well

#50

Post by mjburgess »

s_brady wrote: post #47
mjburgess wrote: post #36

Earlier quoted context omitted.

I'm not exactly sure what you mean. The conditions for causal inference being possible are pretty clear and have to do with the intentional modification of the local environment. The intention to achieve some new environmental state, and your action to bring it about, is a dynamical activity that enables "deep" model building. Causal inference is not going to be some "module" of the brain... it requires a body. When…

Your comment is interesting but seems to conflate two separate though related areas. The need for a body arises from the embodied cognition school of AI which suggests that intelligence is fundamentally embodied, hence the need for a robot equivalent to a body for truly understanding language. However this does not necessarily have to be related to causality, and counterfactual statements about a causal model. The ma…

I'm conflating them because they are deeply connected.

Causal analysis can be performed, via Pearl, on datasets collected for causal analysis.

You still need some mechanism to collect the data, ie., the scientist. This requires solving the "relevance" (/framing) problem -- which, in my view, cannot be solved under a congitivist (/computational) theory of mind.

"Data" which is relevant to a causal hypothesis isn't selected via inference, the body "selects" it.

eg., when my hand is on a hot surface, it's temperature isnt "chosen as the relevant casual variable".

The body is the primary solution to the relevance problem. So you can't just "shove in causal math" into a computational system and expect it to grasp anything.

I also don't think bootstrapping will take you very far: causal models of, eg., dogs are very deep. ie., we understand their 2d, 3d, skeletal, behavioural, color, sound etc. "dimensions".

To say, "the dog was well behaved" requires an extraordinarily deep model of "dog".

The only way i see this being built is via play, ie., via hypothetical interaction with an environment -- as we do -- with bodies capable of discerning relevance.

Post reply on HN