131–140 of 193 posts
The failing seems to be to do with drawing the system boundary in the wrong place (at the walls of the Chinese Room).
An armchair effort to redefine the goalposts and judge NLP, but proof is in the pudding. For now, large language models are the best flavor. NLP models are already useful even in this early stage.
> An armchair effort Given the authors credentials and publication history [1] it's a bit disingenuous to call this an 'armchair effort'. [1] https://scholar.google.com/citations?user=i5sEc1YAAAAJ&hl=en...
This article is....total nonsense. None of the challenges described are impossible to solve with statistical methods. And i'd be willing to bet nearly any amount of money that they will be solved that way, long before they are solved in any other way. The problems posed here are being addressed as we speak with knowledge graphs and graph neural networks, and some of the others can be addressed with more nuanced and c…
What this discussion is missing is Chomsky's distinction between I-language and E-language. I (individual/internal/intensional) language is the knowledge of an individual native speaker. E-language represents the body of external knowledge about language such as corpus data and mass statistical models. Study of I-language looks at idealised individual speaker's internal language capacity, which can generate an infini…
I can see how AI wouldn't learn that from reading/listening to humans. ot would have to learn for itself with reinforcement learning.
I wouldn't call thst language though. But I couldn't think of any kind of non-external knowledge that can be described as language?
This article is....total nonsense. None of the challenges described are impossible to solve with statistical methods. And i'd be willing to bet nearly any amount of money that they will be solved that way, long before they are solved in any other way. The problems posed here are being addressed as we speak with knowledge graphs and graph neural networks, and some of the others can be addressed with more nuanced and c…
Earlier quoted context omitted.
Theoretically, if we had a perfect understanding of human mechanisms of language, we could reproduce it in code. Of course we don't.
I don’t understand how we don’t? I make sound snapping my fingers. Whistling. Why is it so hard to accept mirror neurons fired when early humans heard birds and animals, each other’s grunts and over time we refined it? Everyone has some capacity to refine and strengthen muscle. Why do we need some abstract meta-construct to explain where language comes from? It comes from us. Fleshy meat bags that mutate state over a…
Earlier quoted context omitted.
This seems to be misunderstanding what PAC learnable means. It's saying that for parameters epsilon and delta, you can get an algorithm for learning a hypothesis that, with probability 1 - delta, has an average error rate less than epsilon, given some number of training samples polynomial in 1/delta and 1/epsilon. This has nothing to do with the mental model. The language model doesn't have to spit out probabilities.…
There's a very important difference, though: humans can and often do qualify their interpretation. One major flaw of current PAC-based models is the inability to express certainty and being able to present the rationale behind their (lack of) confidence. One crucial aspect of human language processing is the ability to question or interrogate oneself to adjust one's interpretation. During a dialog this can be done by…
–––––––––––––––––––––––––––––––––
[This is a question and answer series intended to demonstrate the logical reasoning capabilities of an AI system. The AI can understand ambiguity and will ask for more detail in such cases.
Q: What's the capital of Georgia?
A: That's ambiguous. Do you mean the country or the US state?
Q: If Albert is a pig and all pigs are blue, what colour is Albert?
A: Albert is a blue pig.
Q: If Albert is a pig and all pigs are blue, is Albert happy?
A: There is no logical connection between Albert's happiness and his color.
Q: If Alice is a dog and Bob just finished high school, can they have kids together?
A:]{ No.}
[Q: Why can't they have kids together?
A:]{ Because Alice is a dog and Bob just finished high school.}
[Q: Why is that a problem?
A:]{ Because Alice is a dog.
Q: Why is that a problem?
A: Because dogs can't have kids with humans.
Q: Why can't dogs have kids with humans?
A: Because dogs are animals and humans are humans.
Q: Why are humans different from dogs?}
So does it understand after all? Or is trying different prompts a form of p-hacking here?
Earlier quoted context omitted.
My h index is twice as high and that's a terrible h index still lol
This person's h-index is 10x yours: https://scholar.google.com/citations?user=OHbcFkcAAAAJ&hl=en... ! Source: https://www.nature.com/articles/d41586-019-02479-7
Caveat lector: I'm analysing this as a linguist. Not as a programmer. (Unless you count a bunch of bash scripts as "programming".) >Let us start first with describing what we call the “missing text phenomenon” (MTP), that we believe is at the heart of all challenges in natural language understanding. The missing text is not the heart of the challenge; it's just a surface issue. The actual problem is deeper - machines…
GPT-3 can figure out a nonsensical word and use it in the same example. The prompt is between quotes, everything else is generated. > "I got a cat. That wug sleeps on my bed." On every night. He’s really nice. But he isn’t small. > I got a dog. That wug sleeps at my house. On every night. She’s really nice. But she isn’t small. ~~~ Another one in dialogue format. > Human: I got a cat. That wug sleeps on my bed. > AI:…
Earlier quoted context omitted.
I'm sad that statistical methods have gained so much ground over more formal and logical methods (edit: maybe I should instead say "that formal and logical methods have lost so much ground compared to statistical methods"), and, while I can see ways to construct examples that the statistical methods ought to have trouble with, I also notice how incredibly well they've done and how many barriers they've blown past in…
The main problem with using stat models or approaches such as deep learning is not that we are unable to do it. Though possibly trivial, the real problem is we are unable to understand how or why it works which can lead to unintended consequences or lack of ability to support/continue further development (aside from not being able to leverage the new fundamental understanding and apply it to related fields). Imagine…
The issue is that obtaining this understanding requires a lot of effort.
OpenAI invested in trying to understand how a simple CNN works.
You can read their results here: https://distill.pub/2020/circuits/
> In the original narrative of deep learning, each neuron builds progressively more abstract, meaningful features by composing features in the preceding layer. In recent years, there’s been some skepticism of this view, but what happens if you take it really seriously?
> InceptionV1 is a classic vision model with around 10,000 unique neurons — a large number, but still on a scale that a group effort could attack. What if you simply go through the model, neuron by neuron, trying to understand each one and the connections between them? The circuits collaboration aims to find out.
Earlier quoted context omitted.
> A machine does not do that; instead it'll give words intrinsic, non-negotiable meanings. It won't have "wug" in its dataset, so it won't understand that sentence. > I don't believe that machine learning will solve this issue _on its own_, but it could once we're able to simulate that negotiation of meanings. Fascinating example. Personally, I don't reckon there's any inherent reason that forces all machines to give…
Who cares if the machine understands wug? Maybe the issue is more to do with trying to teach language learning as a end on its own. Nobody studies language without purpose, there’s always a next step. If you’re friend is telling you about their cat and what a wug it is, the goal is to conversate, make the other feel heard, entertain them, kill time until something next happens, impress them, and so on. Most of these…