Live data from Hacker News

A glitch in an online survey replaced the word 'yes' with 'forks'

pewresearch.org

31–40 of 102 posts

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#31
post #5

I'm wondering where the "forks" translation came from in the first place. Google Translate used to be fairly reliable for simple translations, but I've seen several examples in the last couple of years where it goes batshit crazy, including starting to loop hallucinating sentences on repeat. Is absolutely nobody checking how well it performs before deploying nowadays?

If you have a plan for automated checking the output of Google Translate against all possible character input strings, be sure to mention it to your recruiter as part of the hiring process.

More seriously: Google Translate's bread-and-butter data source is documents that were human-translated from one language to another with high reliability (such as UN publications). That turns out to work remarkably well for building a neural net that can extrapolate how one sentence should translate to the same sentence in another language. But like most neural networks, it's vulnerable to garbage-in, garbage-out: much like you can get an animal detector to hallucinate "zebra!" if you feed it a noise-pattern as input, if you feed it character sequences that aren't actually words in the input language, it'll try to extrapolate what reality should be between all the corpus it's seen and you'll get garbage on the output side.

Since the tool doesn't actually know what words mean, it has no way, at present, to know "Yes" isn't a Spanish word (and as other commenters have mentioned, it may actually be "a Spanish word" in one weird context in one weird document somewhere in the corpus of all translated documents accessible from the Internet... Or some doc somewhere contains a close-enough typo in the Spanish input document that is over-reflected in the output because no other document contradicts the typo's apparent translation).

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#32

I run an app (Aerolith.org) for studying Scrabble words. For years I would get weird bug reports ( https://github.com/domino14/Webolith/issues/331 ), where at the end of each round, when a user viewed the definitions, the words for definitions would get randomly replaced with other unrelated words. I double and triple checked the code; since I had recently moved the word-related logic to a Go microservice I assumed I…

There is a way to explicitly specify the language of the document in the HTML metadata: ``.

Google doesn't guess if it has strong signal on what the language is.

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#33

So basically Pew is using garbage adware that pops up an intrusive popup that screws up automated language detection, Chrome is way too overeager to auto translate, and Google Translate is starting to go to seed now that everyone uses LLMs.

Google Translate turning a non-word in a source language into an unrelated word in the target language is an old, known misbehavior. I don't think LLMs changed anything there.

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#34
post #19

So basically Pew is using garbage adware that pops up an intrusive popup that screws up automated language detection, Chrome is way too overeager to auto translate, and Google Translate is starting to go to seed now that everyone uses LLMs.

The article says the popup which caused this issue was used to display survey instructions, not ads.

They're probably using an ad network to emit spot-surveys where the ads would show up, although details unclear from the article.

(... honestly, how else will they gather survey data? It's not like they can harass people in the malls Americans don't go to anymore).

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#35

I've had a similar experience where Dutch surnames were translated from English to Dutch by Excel for some reason. Since many Dutchman have a surname prefixed with the Dutch word "van" (which means "of") Excel dutifully translated it to "Busje", which meant that many of our clients suddenly were called "Lieke Busje Lexmond" or "Vincent Busje Gogh". It got a chuckle from our marketing department which caught the error…

Must've been high profile indeed if both Vincent van Gogh and Lieke van Lexmond were there

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#36
post #10
post #6

FWIW, "forks" is "tenedores" in Spanish... not that that illuminates the mystery.

Spanish youngster wants to learn English, and finds the office of a language school. He enters, finds the office of the principal, and knocks on the door. After a short while, he hears: "If, if, between, between!"

This unfortunately reminds me of the level of english guidance my "teacher" would provide at 5th grade down here in Brazil. At least I learned to use a dictionary.

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#37
post #16

Last year there was a question in a foreign language on the trivia league LearnedLeague that had to be thrown out because some peoples' browsers ended up translating the question, making it much easier than anticipated for a subset of users, without them opting in or necessarily even being aware of it.

I remember this one! The question was:

“ Un molcajete y tejolote se usa comúnmente en los restaurantes mexicanos para preparar ¿qué plato popular de aguacate que se suele servir con totopos?”

The idea is that a monolingual English speaker might recognize enough words here to make a guess. Or that people might know Spanish. (I’m somewhere in between, my browser did not translate the question, and I got it right.)

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#38
post #10
post #6

FWIW, "forks" is "tenedores" in Spanish... not that that illuminates the mystery.

Spanish youngster wants to learn English, and finds the office of a language school. He enters, finds the office of the principal, and knocks on the door. After a short while, he hears: "If, if, between, between!"

There's a joke in Portuguese where "entre, meu bem" is translated to English as "between, my well" (in context it should be "come in, my dear").

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#39

I've had a similar experience where Dutch surnames were translated from English to Dutch by Excel for some reason. Since many Dutchman have a surname prefixed with the Dutch word "van" (which means "of") Excel dutifully translated it to "Busje", which meant that many of our clients suddenly were called "Lieke Busje Lexmond" or "Vincent Busje Gogh". It got a chuckle from our marketing department which caught the error…

A colleague of mine got a Christmas card from Microsoft adressed to "New Year's Eve"...

Hi Silvester!

Re: A glitch in an online survey replaced the word 'yes' with 'forks'

#40

> For some respondents, this prompted their browser to believe our survey was written in a language other than English (even though, again, it was in English) and ask if they wanted the page to be translated to English – or, we think, automatically try to translate the page to English. I goddamn hate this "feature" so much. Especially since it sometimes resets and then I have to find out where the fuck Google moved t…

You're going to love YouTube automatically translating video titles to your geo-ip language...

For the next project-manager-promotion-worthy feature, they also do auto-dubbing of the speech now!

Post reply on HN