51–54 of 54 posts
It seems like restrictions on the model talking about non famous people might have been responsible for the appearance of the models being unable to do this.
Earlier quoted context omitted.
But wouldn't every one of those multiple answers be the correct one in this case? Like it can say child a or child b or child c (hypothetical) and while there are mutiple answers, each of them is a logically right one for the question "Who is her child?" no? So how do we judge what is the absolute right answer to that? its ambigious when you say child
Zooming out to the original complaint that "A is B" doesn't imply "B is A" in common English, and then further -- to the goal of having an LLM predict tokens that map closely to truth/logic/helpfulness: I don't think a person speaking plain English in most contexts should be seen as "correct" to answer the question with a non-list answer, even if the question is shaped to expect one, unless there's an established con…
Earlier quoted context omitted.
Pretraining could be reasonably expected to make it learn that mother/father and son/daughter are inverse relationships and Tom is usually a male name.
I'd argue that that's not an easy task in and of itself, but even if someone adds a special exception, there's still the issue that there are many other types of inverse relationship that we understand, but a machine that's just doing pattern matching can't be expected to understand. For instance "boss" and "employee". For instance "waiter" and "customer". For instance "manager" and "player" (in a football context) o…
Earlier quoted context omitted.
This is obviously not as symmetrical as the initial problem, but yes you are expected to be able to easily answer the second after you read the first in a text. That is the concept of quite a lot of early secondary education level tests and also used when learning another language.
But again, you can only make that determination when you know that Oman is a country. LLM's don't know this as a fact, even if they're able to regurgitate a sentence that states this.