151–160 of 211 posts
>If you start a query with “Mary Lee Pfeiffer”, you’re not going to get very far because neural networks aren’t equidistant grids of points (besides the fact that she may not appear very often under that version of her name.) They’re networks of nodes, some with many connections, some with few. One of the ways you optimize large models is by pruning off weakly connected regions. This may come at the expense of destro…
Earlier quoted context omitted.
Interesting; your message hints at a sharp gap between using language stochastically (eg being able to repeat what you've heard) vs using language to express deeper knowledge/understanding. A common argument is that LLM by their nature only model the former, not the latter.
My point is that LLMs can do both, the same way humans can. We don't learn or recall reversals automatically either - we memorize them separately, or run an inference step (which, done often enough, leads to memorization). Reversals aren't free - not for LLMs, and not for our own minds.
This suggests that you may need to train an LLM on its own output. Goes against the conventional wisdom!
I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage,…
The fact is that with zero shot the models actually aren't learning in the sense of updating their knowledge structure, they are instead using background knowledge as inference. We have started calling it zero-shot learning but... well it isn't really.
I fall somewhere on the skeptic side of the LLM spectrum. But this "flaw" just does not seem to have the force that its proponents seem to think it does, unless I'm missing something significant. Simply because in the context of natural language (Rather than formal logical statements), "A is B" does not imply "B is A" in the first place. "Is" can encompass a wide variety of logical relationships in colloquial usage,…
Earlier quoted context omitted.
My point is that LLMs can do both, the same way humans can. We don't learn or recall reversals automatically either - we memorize them separately, or run an inference step (which, done often enough, leads to memorization). Reversals aren't free - not for LLMs, and not for our own minds.
Yes the difference is that LLMs learn statically: they train and that’s it. Humans dynamically: they revisit learned facts and synthesize new ones. If you force an LLM-like training process in a human, by doing rote memorization without any exercises, then you get LLM-like results in that human. This suggests that you may need to train an LLM on its own output. Goes against the conventional wisdom!
Earlier quoted context omitted.
Then you don't want an LLM you want something deterministic
There is no opportunity for nuance?
LLM's have a specific architecture that makes them good at approximate retrieval. They have a specific learning process that makes them able to do this over a vast training set.
Doing things like reasoning about facts is possible using either symbolic or specific sub symbolic structures, but combining these with an LLM is a bit tricky. It's something that could be done for a demo in a 6hr/day/week project (depends on the demo) but to do it for real in an application... or as a robust bit of science... well... 6 people for 6 mths/years/?
Earlier quoted context omitted.
It says "geschäftsführender Bundeskanzler". Even on https://en.wikipedia.org/wiki/Chancellor_of_Germany he's listed as "Vice Chancellor Walter Scheel served as acting Chancellor from 7 May to 16 May 1974" between 4 and 5. There's obviously some ambiguity to it considering we are two humans discussing this with a claimed discrepancy between English Wikipedia and German Wikipedia, but your conclusion is still that the…
There is zero ambiguity under German as law as to who was a Chancellor and who was not. Not a matter of wikipedia etc. The way to be Chancellor is through article 63 of the Grundgesetz, while Scheel was put into the caretaker role via article 69. This explains it a bit https://de.wikipedia.org/wiki/Vizekanzler_(Deutschland) - Scheel was only taking on the function, not the office. This kind of giving some machine the…
When you need to bring up article 63 and 69 of the Grundgesetz to prove that the claim is ludicrous, maybe the reasonable thing to say instead is "I understand why you might think that".
Earlier quoted context omitted.
> Scheel became acting Chancellor > Chancellor of Germany, Acting, 7 May 1974 – 16 May 1974 https://en.wikipedia.org/wiki/Walter_Scheel
I know, but the German wikipedia gets it right: https://de.wikipedia.org/wiki/Walter_Scheel (And so does the German Chancellor: https://www.bundeskanzler.de/bk-de/kanzleramt/bundeskanzler-... ) There is zero ambiguity about who was Chancellor and who was not.
Earlier quoted context omitted.
I know, but the German wikipedia gets it right: https://de.wikipedia.org/wiki/Walter_Scheel (And so does the German Chancellor: https://www.bundeskanzler.de/bk-de/kanzleramt/bundeskanzler-... ) There is zero ambiguity about who was Chancellor and who was not.
So the English Wikipedia author is a moron, or is there ambiguity when describing the role in English?
So either need to be fully descriptive (e.g., something like fulfilling the functions of the Chancellor, while not ever having the office) or it will be also open to being misunderstood.
The issue here really is that the German succession doesn't ever transfer the office, but only the function (which is different to the US, for example). So here Scheel followed Brandt, but not into the office. Only someone having the office is a Chancellor and there is a specific way to that office.