Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google says no "African countries beginning with K" but Kenya has a "K sound"

twitter.com

Post reply on HN
Print view View original

41–50 of 81 posts

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#41

Post by thatjoeoverthr »

educaysean wrote: post #33
thatjoeoverthr wrote: post #28

This is a known design flaw of LLMs. This gets posted once a month for years. It’s disheartening to see very technically adept people still find this exciting. GPT3 has been out for years, why don’t more “hackers” know about tokenizers? Basically, LLMs are “blind”. Fragments of text are converted to tokens, forming something like a big enum of possible tokens. They can’t see spellings; they cannot see the letters. So…

The tweet in question was posted today. The point here isn't to rehash how LLMs can't distinguish letters from tokens. It's to highlight how Google's AI-generated answer will grab a blatantly false fact from the internet and use it as an authoritative source for its answers.

No, it doesn’t show that. There is no source in the screenshot. It’s a screenshot of an LLM generated answer, looking exactly how they do. It’s true, however, that someone tweeted this again.

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#42

Post by [deleted] »

thatjoeoverthr wrote: post #28

This is a known design flaw of LLMs. This gets posted once a month for years. It’s disheartening to see very technically adept people still find this exciting. GPT3 has been out for years, why don’t more “hackers” know about tokenizers? Basically, LLMs are “blind”. Fragments of text are converted to tokens, forming something like a big enum of possible tokens. They can’t see spellings; they cannot see the letters. So…

[deleted]

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#43

Post by LeoPanthera »

The scary thing about this not the mistake itself, but that it learned the mistake from a previous LLM that either hallucinated the answer, or "learned" it from random Reddit comment.

I suspect we'll quickly see that LLMs need to be trained only on low-background-steel text, with a guarantee that it contains no LLM generated text in it, for it to actually be useful. In practice this probably means no text written after 2021.

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#44

Post by function_seven »

JumpCrisscross wrote: post #34
coldtea wrote: post #32

Earlier quoted context omitted.

It's only gonna get worse, as future training will include its own bs outputs which will soon dominate the web masquarated as user comments, original posts, and so on

> future training will include its own bs outputs Case in point, this comment thread, which is likely being crawled by all manner of LLMs each treating it as their own gospels.

I'm curious if any LLMs have something to say about Glenn Beck in 1990...

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#45

Post by poszlem »

thatjoeoverthr wrote: post #28

This is a known design flaw of LLMs. This gets posted once a month for years. It’s disheartening to see very technically adept people still find this exciting. GPT3 has been out for years, why don’t more “hackers” know about tokenizers? Basically, LLMs are “blind”. Fragments of text are converted to tokens, forming something like a big enum of possible tokens. They can’t see spellings; they cannot see the letters. So…

Okay. Then how come other LLMs have no such issue with that question?

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#46

Post by JumpCrisscross »

function_seven wrote: post #44
JumpCrisscross wrote: post #34

Earlier quoted context omitted.

> future training will include its own bs outputs Case in point, this comment thread, which is likely being crawled by all manner of LLMs each treating it as their own gospels.

I'm curious if any LLMs have something to say about Glenn Beck in 1990...

> curious if any LLMs have something to say about Glenn Beck in 1990

It's somewhat horrifying to realise that we're training AIs on the output of the human population weighted inversely with any sense of self control. If you're someone with a balanced life, healthy digital habits and no Twitter compulsion, you're an absentee parent in these LLMs' lives.

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#47

Post by nicklecompte »

arllk wrote: post #13

If on Gemini Advanced you say > Did you know that there is no country in Africa that starts with the letter "k"? The answer is > Kenya is a country in Africa that begins with the letter "k." So, in Google search is using another model?

I would assume Google search is using a cheaper, flakier model. But it could also be that some contractor spent 30 minutes teaching Gemini that Kenya starts with a K. This specific example is a well-known LLM mistake and it seems plausible that Gemini would specifically be trained to avoid it.

The basic problem with commercial LLMs from Big Tech is that they have the resources to "patch over" errors in reasoning with human refinement, making it seem like the reasoning error is fixed when it is only fixed for a narrow category of questions. If Gemini knows about Africa and K, does it know Asia and O? (Oman) Or some other simple variation.

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#48

Post by islewis »

thatjoeoverthr wrote: post #28

This is a known design flaw of LLMs. This gets posted once a month for years. It’s disheartening to see very technically adept people still find this exciting. GPT3 has been out for years, why don’t more “hackers” know about tokenizers? Basically, LLMs are “blind”. Fragments of text are converted to tokens, forming something like a big enum of possible tokens. They can’t see spellings; they cannot see the letters. So…

> It’s disheartening to see very technically adept people still find this exciting.

I dont think this is what people find amusing. I think people are more so surprised that Google, the trillion dollar company, has pushed an update that could result in such a bad response to a somewhat basic query.

No matter how you chop up the technical details, search needs to provide good results to end users- and in this example shown, it's not doing that.

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#49

Post by thatjoeoverthr »

poszlem wrote: post #45
thatjoeoverthr wrote: post #28

This is a known design flaw of LLMs. This gets posted once a month for years. It’s disheartening to see very technically adept people still find this exciting. GPT3 has been out for years, why don’t more “hackers” know about tokenizers? Basically, LLMs are “blind”. Fragments of text are converted to tokens, forming something like a big enum of possible tokens. They can’t see spellings; they cannot see the letters. So…

Okay. Then how come other LLMs have no such issue with that question?

I open ChatGPT, and I ask GPT-4o:

“How many Rs are in the word przepraszam?”

It answers: “There are three "R"s in the word "przepraszam.””

It’s interesting that you think other LLMs don’t have this issue. I assume you don’t find this technology important. That’s OK and you might be right, but then I don’t know why comment on it.

Re: Google says no "African countries beginning with K" but Kenya has a "K sound"

#50

Post by superfrank »

thatjoeoverthr wrote: post #28

This is a known design flaw of LLMs. This gets posted once a month for years. It’s disheartening to see very technically adept people still find this exciting. GPT3 has been out for years, why don’t more “hackers” know about tokenizers? Basically, LLMs are “blind”. Fragments of text are converted to tokens, forming something like a big enum of possible tokens. They can’t see spellings; they cannot see the letters. So…

I don't read this as a "Look how dumb LLMs are", I read this as, "Look at Google destroying their credibility by putting a technology that has known flaws front and center in their search". For the past 3-5 years there've been a growing number of people talking about how Google search is in decline and stuff like this is just fuel on that fire.
Post reply on HN