Live data from Hacker News

Claude 3 beats Google Translate

arxiv.org

101–110 of 127 posts

Re: Claude 3 beats Google Translate

#101

Earlier quoted context omitted.

This is why for things that matter, human translators will always have a job.

That's a big leap of faith from one example to "always"

Maybe they're not more accurate but more for assignment of responsibility. Just from my experience of legal system, official documents and i8n

Re: Claude 3 beats Google Translate

#102
post #20

Earlier quoted context omitted.

What are you even talking about? You don’t just “use a gpu.” Software doesnt get better by magically throwing it at a gpu. Moreover you can’t just run any old software on a gpu, it has to be built for it. Even then, the gpu is just a speed increase, not a magical make better box. That’s like saying any random text editor would be a better translator if they ran on GPUs. EDIT: google also literally builds its own acce…

No... I mean LLM's use a GPU. Hence why the cost is different. Changing Google Translate to use an LLM would become much more expensive for Google. This is why you see a different cost structure for using "AI". LLM's can't realistically use the CPU for anything serious.

You have it backwards. For any given amount of compute needed, GPU's are a lot cheaper than CPU's. But GPU's can't run all the code you can run on a CPU.

The reason models that use GPU's cost more to run, is that they tend to be A LOT more compute intensive. However, if you run the inference for the same models on CPU, they will be both much more expensive than on GPU's (or on specialized tensor silicon) and also slower.

Re: Claude 3 beats Google Translate

#103

Earlier quoted context omitted.

Where is the hallucination? It seems in line with the others.

You think this: "Swarming like a swarm of bees. He was carried among the people, hanging from the handle. No matter how good you think about the situation you're in, it's disgusting. Where are you now?" is a comparable translation to this: "No matter how you couch it, riding the subway feels disgusting: you dangle like ripe fruit from a hanging vine, squeezed in among humans swarming like bees." Or this ?: "Being cra…

Using metaphorical or allegorical language as a test isn't that useful. Getting something appropriate is going to be much more up to chance.

Re: Claude 3 beats Google Translate

#104
This doesn't surprise me, Claude 3 beats pretty much anything I use it for.

Still, my wife and I translate erotica at times, and both Claude and GPT-4 refuse to translate a shit ton of content, while Google Translate and DeepL will happily translate anything we throw at them.

I wonder if an uncensored version of Llama 3 would perform better. It's supposed to be on GPT-4 level in certain languages, after all.

Re: Claude 3 beats Google Translate

#105

This doesn't surprise me, Claude 3 beats pretty much anything I use it for. Still, my wife and I translate erotica at times, and both Claude and GPT-4 refuse to translate a shit ton of content, while Google Translate and DeepL will happily translate anything we throw at them. I wonder if an uncensored version of Llama 3 would perform better. It's supposed to be on GPT-4 level in certain languages, after all.

Yes, Claude may make advances here and there in it's tech. But it's main feature is that it is censored (or "safe). That will forever be Claude's main selling point: censorship.

Any technological gains it makes will just be marketing fodder to get their pre-captured AI into more people's stack.

They are lucky they haven't tried images yet.

Re: Claude 3 beats Google Translate

#106
I assume that Google Translate has a much larger usage volume than any of the free-to-use LLMs.

I don't know the average energy/hardware*time usage per query on google translate vs competing LLMs such as Claude 3 Opus but I wouldn't be surprised that a large LLM such as Claude 3 Opus would be much too expensive to be used as the backend model for a free service like Google Translate.

The paper authors do acknowledge this concern and run experiments on smaller models with knowledge distillation. However, as far as I know we cannot know if their distilled networks can compete with the current Google Translate system in terms of energy / hardware usage efficiency.

Re: Claude 3 beats Google Translate

#107
Neither Google translate or DeepL seem to be good at translating Japanese to English, so this is hardly a surprise to me.

In the case of Japanese, only an LLM seems to be capable of tracking the gender of a fictional character, and since gender is rarely indicated in the original japanese language, Google Translate will alternate every sentence indicating whether "He" or "she" took action when referring to the same character.

This is just the tip of the iceberg in the problems that come up when trying to translate Japanese. LLM's on the other hand have awareness of the "content" -- they understand what is happening in the original story and it auds their translation choices -- and LLMs tend to be superior at novel translation in general.

I imagine the non LLM tools work much better translating between similar languages.

Re: Claude 3 beats Google Translate

#108

Earlier quoted context omitted.

You think this: "Swarming like a swarm of bees. He was carried among the people, hanging from the handle. No matter how good you think about the situation you're in, it's disgusting. Where are you now?" is a comparable translation to this: "No matter how you couch it, riding the subway feels disgusting: you dangle like ripe fruit from a hanging vine, squeezed in among humans swarming like bees." Or this ?: "Being cra…

Using metaphorical or allegorical language as a test isn't that useful. Getting something appropriate is going to be much more up to chance.

>>>Using metaphorical or allegorical language as a test isn't that useful.

Because nobody wants to translate literature with metaphorical language?

Re: Claude 3 beats Google Translate

#109
I read the article and I found it quite lacking. Why on Earth would you force your LLM to translate sentence by sentence? It ruins the whole interest of LLM, which is to use large contexts to drive your generation. I used Deepl a lot in the past and I had a recurrent problem when translating from French into English, computer related texts. In French, a "chaine" in the context of computer science is mostly translated as "string", however, when translating with Deepl (or Google translate) since the model would not take previous sentences into account, the system would loose the computer context and translate "chaine" into "chain", which of course was usually wrong.

But the funniest part was when I wanted to translate "jeûner" in Greek. "jeûner" in French means "to fast", in the sense of not eating. However, Google translated "jeûner" into "gregoria" in Greek, which means fast in the sense of speed... It went through English to translate "jeûner" into "fast" then "fast" into "gregoria"...

Re: Claude 3 beats Google Translate

#110

Im surprised they didn’t try out Gemini. It’s a lot better than Google translate and in my experience this is one use case where Gemini outshines even other frontier models like ChatGPT4

Maybe the backend uses more energy compared to Google translate?

Maybe but it just seems silly to not briefly try out a few of the options as a teacher model before settling on one (Claude)
Post reply on HN