Claude 3 beats Google Translate • Hacker News Board

Re: Claude 3 beats Google Translate

#31

Post by frabcus » Tue, Apr 23, 2024, 12:41 PM UTC

sunaookami wrote: post #4

DeepL already beat Google Translate years ago.

In tests I did with linguists a year ago, GPT-3.5 was better than DeepL also. Google Translate largely caught up with DeepL a while ago!

Specifically, LLMs have a context window larger than a sentence. So can eg infer gender throughout documents. Neither DeepL or Google Translate did that.

Re: Claude 3 beats Google Translate

#32

Post by eptcyka » Tue, Apr 23, 2024, 12:42 PM UTC

deepvibrations wrote: post #26

Pretty much all LLMs easily beat Google Translate, this is not news... One of the best yet unplanned features of them imo.

But it does. It offers "Rēdze tapa" as one of the alternative translations of "wrist" when translating from English to Latvian. There is no such concept as a "rēdze tapa" in Latvian. It might be a byproduct of translating English first to russian and then to Latvian.

Re: Claude 3 beats Google Translate

#33

Post by SamBam » Tue, Apr 23, 2024, 12:43 PM UTC

Tistron wrote: post #3

ChatGPT 3 was already far beyond google translate in my attempts, mostly using it between English, Danish and Swedish. Even being able to ask it to translate song lyrics while retaining rhyming structure was sometimes not too bad :)

Also being able to ask it to use the formal vs. informal is very helpful for romance languages. I could never get Google translate to do that (although sometimes adding ", sir" to the end of a sentence would trick it into using the formal).

Re: Claude 3 beats Google Translate

#34

Post by paxys » Tue, Apr 23, 2024, 12:45 PM UTC

deepvibrations wrote: post #26

Pretty much all LLMs easily beat Google Translate, this is not news... One of the best yet unplanned features of them imo.

It's not unplanned. The transformer architecture was literally created for better language translation.

Re: Claude 3 beats Google Translate

#35

Post by jsheard » Tue, Apr 23, 2024, 12:45 PM UTC

frabcus wrote: post #27

endisneigh wrote: post #17

Earlier quoted context omitted.

Orders of magnitude more energy necessary though.

The GPT 3.5 API is cheaper than Google Translate. And in our testing (over a year ago) was better for translation. So I assume in that case the energy use is less?

Isn't OpenAI still operating at a loss? You can't infer much from an API bring cheaper if it's being subsidised.

Re: Claude 3 beats Google Translate

#36

Post by magoghm » Tue, Apr 23, 2024, 12:46 PM UTC

Alifatisk wrote: post #9

herculity275 wrote: post #8

Earlier quoted context omitted.

Didn't "Attention Is All you Need" bill transformers primarily as a translation model?

Yes, it was primarly for translation. I don’t know how OP come to the conclusion that it was accidental

I guess that what OP meant is that LLMs are not specifically trained to do translation, they learn to do it as a side effect.

Re: Claude 3 beats Google Translate

#37

Post by ojosilva » Tue, Apr 23, 2024, 12:46 PM UTC

Slightly (un)related, but there's an interesting recent paper on how LLMs perform on code translation:

https://arxiv.org/pdf/2308.03109.pdf

Re: Claude 3 beats Google Translate

#38

Post by yau8edq12i » Tue, Apr 23, 2024, 12:46 PM UTC

londons_explore wrote: post #24

snapcaster wrote: post #5

Yeah this is old news. When i was in China last year google translate seemed to literally never work (looks of confusion trying to do basic interactions in stores). GPT-4 worked perfectly every time, I think google translate might be another soft abandoned project from google

> I think google translate might be another soft abandoned project from google There are 100+ people working on Google translate and associated stuff (the mobile apps, the serverside stuff, etc). I guess they're all asleep.

Where did you get this number from?

Re: Claude 3 beats Google Translate

#39

Post by casey2 » Tue, Apr 23, 2024, 12:47 PM UTC

sunaookami wrote: post #4

DeepL already beat Google Translate years ago.

DeepL is also very bad, it ties with google in plenty of very basic sentences. Try out: 私は毎週本を読む。友達は漫画だ。 Meta's gets it despite a message claiming that it doesn't know non-English languages very well (which is true).

Re: Claude 3 beats Google Translate

#40

Post by famouswaffles » Tue, Apr 23, 2024, 12:47 PM UTC

maven29 wrote: post #18

People trust Google Translate to not go off-topic or "hallucinate" and there is still a lot of value in something semi-deterministic like that. (Edit: To clarify, maintaining semantics is held sacrosanct in classical methods of machine translation)

>People trust Google Translate to not go off-topic or "hallucinate"

Then "people" have no idea what they're doing. Google Translate and Deepl "hallucinate" way more than the likes of GPT-4 and Claude 3 for Translation.