Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We let models localize into 16 languages. How we made it read native.

reelang.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: We let models localize into 16 languages. How we made it read native.

#11

Post by stuess »

tetromino_ wrote: post #9
applfanboysbgon wrote: post #4

It is absolutely revolting to see LLM translation being sold as "better than professionals". Yeah, yeah, I saw you attached a caveat to it. No. You don't need to be dishonest. LLM translation is shit but it's borderline free, and you want the benefit of translation without having to pay for it. Who can argue with the economics of "free", after all? I would leave it at that, but then I consider what you're actually us…

> LLM translation is shit Compared to who? Compared to a professional with a language degree, with a feel and love for both languages and an understanding of the subject area, LLM translation is shit, no question. But such professionals don't get hired by startups to localize their websites, too expensive. When comparing to paid- by-the-word contractors who do website localization, I am not so sure that LLM is worse…

My point here is exactly that, as a startup, there's l10n that we can afford and l10n that's perfect. Could a human that has plenty of time to build context, browse the website, think about and learn our product do a better job at transcreating? They sure would.

But the reality for us (and many other startups) is that the options are no l10n and locking people who don't speak English (well) out of our product or having one that we - at least - do our best to make feel pretty native. We also built a version in Simple English. (I wrote an article sharing that process, too: https://reelang.com/open-startup/blog/how-to-build-a-simple-...)

There is also a reality that coding agents do have the ability to build more context than the average human translator with the average tooling like Smartcat or Weglot would.

Re: We let models localize into 16 languages. How we made it read native.

#12

Post by hidalgoerrant »

applfanboysbgon wrote: post #4

It is absolutely revolting to see LLM translation being sold as "better than professionals". Yeah, yeah, I saw you attached a caveat to it. No. You don't need to be dishonest. LLM translation is shit but it's borderline free, and you want the benefit of translation without having to pay for it. Who can argue with the economics of "free", after all? I would leave it at that, but then I consider what you're actually us…

Language learners need to be able to use the site in their native language. The whole world doesn’t feel comfortable using English as a second language to learn another language. None of my students use English to learn Spanish or Mandarin Chinese. They use their mother tongues so they don’t have to add to their cognitive load by processing another language on top of the one they're already learning.

The localized sites are exactly what they can judge as native speakers. They’re judging copy in their own language, not the language they’re learning.

We noticed that a good share of our learners were already using Google Translate built into their browsers to access the site in their native languages. The quality is atrocious, so what we’re offering is a step change from that.

We offer 58 languages to learners. Only 16 have been localized. That means the chrome of the website and apps -- the learner’s native language -- not the video content they watch in the target language they’re learning.

Between my co-founder and me plus people who have kindly offered to review the site we can judge seven of those localizations well enough to see that they’re orders of magnitude better than what Google Translate would concoct for them.

Re: We let models localize into 16 languages. How we made it read native.

#13

Post by hidalgoerrant »

Anduia wrote: post #5

I'm so confused with the "Upgrade" example. Why does it lose the title case and why does it end up phrased as "upgrade to" instead of "Upgrade"?

English can use "upgrade" as an intentionally underspecified one-word label. On, say, a paywall, Spanish would naturally use a construction like "pasar a [otro plan]," and French “passer à [une autre offre].” The preposition a/à is how that particular meaning is expressed in those languages.

So it isn’t randomly translating “upgrade" as "upgrade to." The context tells you that "upgrade” means "move to a different plan," rather than "update the software."

You're right about the random capitalization though - it wasn't intentional and we'll edit that bit to avoid confusion.

Re: We let models localize into 16 languages. How we made it read native.

#14

Post by stuess »

thaumasiotes wrote: post #6

Huh. This article is about the issue of picking which translation of a word to use according to what the word means in context... a task that LLMs should ace. I was hoping this was about using an LLM to produce text in the house style. That's also a task they should be good at, but that clearly doesn't happen by default. Here's a two-page public announcement from Ironhide Games, who make Kingdom Rush. They're apologi…

This is interesting! Exactly our experience, you do have to guide LLMs on author the source text in the way we would write, with few-shot examples in the prompt. And then prompt them to transcreate so it doesn't calque (read as a literal translation rather than something a native speaker would have written for other native speakers). Both of this can be solved for reasonably well with the right prompt guidance IMHO.

In my experience, there's two key incredients to get to a good results (with humans OR LLMs doing the work), I've worked on plenty of sites that had human l10n before LLMs came about.

If you're interested in the actual prompt we're using for our voice that includes instructions on how to avoid sounding like bad LLM copy, I've popped that here in a Github Gist: https://gist.github.com/tobyurff/2b461c259c34dfc6758a3932982...

Hope it helps, I might turn that into a blog post one day as well, seems like this could be useful to others!

Re: We let models localize into 16 languages. How we made it read native.

#15

Post by stuess »

applfanboysbgon wrote: post #10
tetromino_ wrote: post #9

Earlier quoted context omitted.

> LLM translation is shit Compared to who? Compared to a professional with a language degree, with a feel and love for both languages and an understanding of the subject area, LLM translation is shit, no question. But such professionals don't get hired by startups to localize their websites, too expensive. When comparing to paid- by-the-word contractors who do website localization, I am not so sure that LLM is worse…

My experience with dirt cheap contractors you're talking about is that they simply throw what you put into Google Translate / an LLM themselves. So yes, strictly speaking, doing that yourself is better than paying someone else to do it for you. But it's not better than hiring a real professional, so let's not pretend it is. It is what it is: cheap. Be up front about that. And I think using machine translation, and be…

The point I'm making isn't (in life or in this blog post) that LLMs do a better job than humans. My point is that LLMs can get pretty close with the right context, and in some circumstances they can build context better than the typical human translator would.

Most CAT tools do vet for ML/LLM translations these days, so "cheap" human translation is mostly ML-assisted and human-edited, but my entire point is that they're not to blame, the majority of translation fails happens because of the lack of context, not because the human didn't try. It's about setting the site and l10n infrastructure out in a way that gives them the freedom to translate in the way that makes most sense for their language without being forced into "English in a costume".

It is unfortunately a fact that one can be both things: a startup and a language learning app. But my entire point wasn't about LLMs vs. humans, it was about laying out the architecture in a way that enables great l10n, whether we do that with LLMs today or humans tomorrow once we can afford it.

Re: We let models localize into 16 languages. How we made it read native.

#16

Post by stuess »

applfanboysbgon wrote: post #4

It is absolutely revolting to see LLM translation being sold as "better than professionals". Yeah, yeah, I saw you attached a caveat to it. No. You don't need to be dishonest. LLM translation is shit but it's borderline free, and you want the benefit of translation without having to pay for it. Who can argue with the economics of "free", after all? I would leave it at that, but then I consider what you're actually us…

So our approach here to vet these:

Between the two of us, we do speak 7 languages well enough to judge a localized site, and we have people around us for review on some more.

We run our workflow for a new localized version of say, German, then judge the quality and add rules to our prompt, guidance and voice guide for issues we spot, never hand-editing the actual strings.

We then re-run it until it's in a place where we feel this feels fairly native (feels transcreated, not calqued) and serves its purpose well).

Once we had that done in a good amount of locales, we started running it in languages we can't judge ourselves. Here we do depend on people on the app/website to spot and help us if we got it wrong somewhere, so it's a living process that evolves.

Here's an example of how that guidance and prompt evolved over time, in case you're interested: https://gist.github.com/tobyurff/2b461c259c34dfc6758a3932982...

Re: We let models localize into 16 languages. How we made it read native.

#17

Post by applfanboysbgon »

stuess wrote: post #15
applfanboysbgon wrote: post #10

Earlier quoted context omitted.

My experience with dirt cheap contractors you're talking about is that they simply throw what you put into Google Translate / an LLM themselves. So yes, strictly speaking, doing that yourself is better than paying someone else to do it for you. But it's not better than hiring a real professional, so let's not pretend it is. It is what it is: cheap. Be up front about that. And I think using machine translation, and be…

The point I'm making isn't (in life or in this blog post) that LLMs do a better job than humans. My point is that LLMs can get pretty close with the right context, and in some circumstances they can build context better than the typical human translator would. Most CAT tools do vet for ML/LLM translations these days, so "cheap" human translation is mostly ML-assisted and human-edited, but my entire point is that they…

> The point I'm making isn't (in life or in this blog post) that LLMs do a better job than humans.

Your literal headline claim is that LLMs are better than humans. With a caveat attached, sure, but you're immediately setting the tone with that.

> It is unfortunately a fact that one can be both things: a startup and a language learning app

You know what you do in that case? Teach the languages you know! It's arrogant and greedy beyond belief to think you have any ability or right to "teach" people 58 languages. If you were genuinely interested in helping people learn and not making a quick AI startup cash grab, you'd start with the language pair you're most fluent in, then if that has any success, slowly expand your offerings once your concept is proven, hiring more people as you can. Of course, such a rational business as that probably doesn't sound as juicy as "we used AI to SOLVE TRANSLATION" to the investors you're hoping to attract...

Post reply on HN