Bard uses a Hacker News comment as source to say that Bard has shut down

Kleenex and Xerox sympathize.

For example, a 'what does this program do' type LLM software (with chatgpt at the backend) was broken by doing little more than changing the name of the function being called from "Q_rsqrt" to "Floor." [1] Even though the rest of the function was something that the software was clearly trained on verbatim and even included an almost certainly unique hex value (as part of bit math games), the weighting of the function's name ended up overriding everything else and so it described, in quite compelling detail, how the function was calculating the floor of the input value. Except it wasn't doing that, at all.

If these tools ever become more than toys, they will be far more vulnerable to the equivalent of SEO than search engines ever were. And it's probably an insurmountable problem, because it's a vulnerability in how they fundamentally operate.

[1] - https://news.ycombinator.com/item?id=35164583

I'm mostly with you, though I suspect if you asked one of these programs to summarize the comment in question, it would not end up attributing any statements to Google.

I don't understand how the source attribution is made; in my concept of what these programs are doing, that information ("where did this idea/text come from?") doesn't really exist in the finished model.

Think of my comment as more directed toward "what other people are saying about this flub" than "how the software should behave".

> If these tools ever become more than toys, they will be far more vulnerable to the equivalent of SEO than search engines ever were. And it's probably an insurmountable problem, because it's a vulnerability in how they fundamentally operate.

On the other hand, I think these tools are fundamentally doing language processing correctly, and most of what's going wrong is that people expect language processing to be able to answer the wrong kinds of questions. (Related: most people believe that their thoughts are expressed linguistically, but this is not true.) I expect future developments in this area to involve harnessing language models fairly similar to these to separate software.

There's an example of this mistake in another current frontpage link: https://twofergoofer.com/blog/bard

> It's particularly fun because the robots don't actually understand what rhyming is.

But "what rhyming is" is something the current set of chatbots can understand perfectly. Rhyming is a division of words, which are the only thing they do understand, into a set (actually, several related sets) of equivalence classes. To give a rhymed answer to a riddle, it's perfectly sufficient to know that two words rhyme; there's no need to know how the rhyming between them works.

And in fact, we have rhyme tables for ancient Chinese characters that give us just this information. Trained specialists are able to tell whether a passage of Old Chinese rhymes or doesn't rhyme. They are generally not able to explain exactly how two characters rhyme; the phonetic information doesn't survive. Does that mean that modern humans "don't actually understand what rhyming is"?