Live data from Hacker News

Claude 3 beats Google Translate

arxiv.org

61–70 of 127 posts

Re: Claude 3 beats Google Translate

#61
post #34

Pretty much all LLMs easily beat Google Translate, this is not news... One of the best yet unplanned features of them imo.

It's not unplanned. The transformer architecture was literally created for better language translation.

Not the decoder-only architecture.

Re: Claude 3 beats Google Translate

#62
post #43

Earlier quoted context omitted.

Google Translate use recent neural net tech, and does hallucinate translations, since years ago. Translating from french to english, sometimes it miss the translations by a lot just because an accent is missing. Stop dissing technology based on your belief. https://en.wikipedia.org/wiki/Google_Neural_Machine_Translat...

I’m curious do you have a reproducible example of this?

While I am also curious, "reproducible" is probably an impossible standard — even if Google sets the temperature to 0 (which I don't think they do, because it shows you other possible translations when you click on the output), they'll be regularly updating (or at least A/B testing) the model.

Re: Claude 3 beats Google Translate

#63
post #52

Earlier quoted context omitted.

I think it was around 2018 when Google Translate translated Chinese 万 (ten thousand) to “million” for me, making the stats I was looking at completely nonsensical. I was shocked it managed be so wrong about something so basic. I wouldn’t call it trustworthy.

This is why for things that matter, human translators will always have a job.

That's a big leap of faith from one example to "always"

Re: Claude 3 beats Google Translate

#64
post #35

Earlier quoted context omitted.

Isn't OpenAI still operating at a loss? You can't infer much from an API bring cheaper if it's being subsidised.

GPT4 on the web is at a loss, $20/mo doesn't cover it unless someone's a rare user. GPT4 through API is not at a loss, and I doubt GPT3.5 is at a loss either.

Given that you can buy a at least a couple hours worth of GPU compute at $20, it's hard to believe the average user spends that much time (specifically waiting for the response) on GPT-4 on the web every month... Not to mention that they probably have optimizations that batch together queries at the same time and make things more efficient at scale...

Re: Claude 3 beats Google Translate

#65

Earlier quoted context omitted.

I’m fascinated by this since I use translate regularly. Do you have an example of google translate doing this?

https://i.imgur.com/e1aeLej.png

Where is the hallucination? It seems in line with the others.

Re: Claude 3 beats Google Translate

#66
post #35

Earlier quoted context omitted.

Isn't OpenAI still operating at a loss? You can't infer much from an API bring cheaper if it's being subsidised.

GPT4 on the web is at a loss, $20/mo doesn't cover it unless someone's a rare user. GPT4 through API is not at a loss, and I doubt GPT3.5 is at a loss either.

the assertions were based on a prompt sent to chatgpt: "are you more efficient than google translate, cost-wise?"

Re: Claude 3 beats Google Translate

#67

I still find it amazing how LLM translation capabilities are an almost accidental feature. Yet they still managed to leapfrog decades of research and billions of investment dollars in traditional machine translation.

Transformers have been built for translation since the very first iteration. 2017 paper.

Re: Claude 3 beats Google Translate

#68
post #3

ChatGPT 3 was already far beyond google translate in my attempts, mostly using it between English, Danish and Swedish. Even being able to ask it to translate song lyrics while retaining rhyming structure was sometimes not too bad :)

[Nit] It's either GPT-3 or ChatGPT (which was also called GPT-3.5-turbo)

I do agree GPT-3 probably did perform better than Google Translate though.

Re: Claude 3 beats Google Translate

#69
post #58

Earlier quoted context omitted.

>People trust Google Translate to not go off-topic or "hallucinate" Then "people" have no idea what they're doing. Google Translate and Deepl "hallucinate" way more than the likes of GPT-4 and Claude 3 for Translation.

Maybe some ppl do know what they are doing? You sound unhinged

If anyone is saying they won't use GPT-4 for translations over Google or Deepl because of hallucinations then it's a clear sign they haven't actually used all and compared.

"I'll use Google because GPT hallucinates" is a hilarious thing to say when Google still regularly devolves to half gibberish on distant language pairs.

Re: Claude 3 beats Google Translate

#70
post #14

Earlier quoted context omitted.

Translation is the reason transformer models exist. Every other use case is basically an accidental feature

The original Transformer, indeed for translation, was an encoder-decoder architecture. Most current LLMs, like Claude 3, are "decoder only". So in a sense the decoder-only translation is kinda accidental feature too.

So anything except encoder decoder translation is accidental? Crazy way to define research.
Post reply on HN