Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fluent 1.0: a localization system for natural-sounding translations

hacks.mozilla.org

Post reply on HN
Print view View original

101–110 of 117 posts

Re: Fluent 1.0: a localization system for natural-sounding translations

#101

Post by ascar »

zbraniecki wrote: post #85
ascar wrote: post #28

Earlier quoted context omitted.

GNU gettext is a well working framework for i18n. For the sentence ordering we include all variables in the translations, but split translations on styles. We then give the translator the text in order of html appearance in source code for context. Translators can then rearrange everything but the variable across the string and also leave stuff blank when necessary. It's not perfect, but works in most cases. And in t…

Hi! We've been working with and evaluating Gettext when we started Fluent. Our opinion is similar to Unicode's - Gettext is fundamentally flawed design for internationalization purposes. Here's you can find more detailed explanation of our position - https://github.com/projectfluent/fluent/wiki/Fluent-vs-gette... Please, don't take it as a criticism of using it. We just don't think it scales and we don't think it's p…

You make a lot of good points in the linked article, but you lose some credibility right from the start

> Secondly, it makes it impossible to introduce multiple messages with the same source string which should be translated differently.

This is false. The gettext message format uses msgctxt to deal with this. It's a fundamental part of the format. The unique identifier is the combination of msgctxt and the singular string. I wonder how you could miss that? We actually use an automatically generated msgctxt for some part of our app to avoid accidentally translating the same source text incorrectly in different context.

Also I couldn't quite follow the point about interpolation of fluent vs gettext (probably because I don't know fluent). Message interpolation in gettext works and can be absolutely readable. E.g. "You have {count} items". The big drawback is that you can't move this variable across strings. Can you do that with fluent?

Re: Fluent 1.0: a localization system for natural-sounding translations

#102

Post by zbraniecki »

ascar wrote: post #101
zbraniecki wrote: post #85

Earlier quoted context omitted.

Hi! We've been working with and evaluating Gettext when we started Fluent. Our opinion is similar to Unicode's - Gettext is fundamentally flawed design for internationalization purposes. Here's you can find more detailed explanation of our position - https://github.com/projectfluent/fluent/wiki/Fluent-vs-gette... Please, don't take it as a criticism of using it. We just don't think it scales and we don't think it's p…

You make a lot of good points in the linked article, but you lose some credibility right from the start > Secondly, it makes it impossible to introduce multiple messages with the same source string which should be translated differently. This is false. The gettext message format uses msgctxt to deal with this. It's a fundamental part of the format. The unique identifier is the combination of msgctxt and the singular…

> but you lose some credibility right from the start

Thank you for the feedback! I updated the article to include the mention about `msgctxt`.

Personally, in my experience, many project environments end up with partial support for this feature (for example many react/angular extractors don't support it) which leads to limited use and requires the localizer to request adding a context by the developer.

I did not include that since it's just my personal experience and I assume more mature projects tend to recognize the feature and use it, hopefully, extensively :)

> Message interpolation in gettext works and can be absolutely readable. E.g. "You have {count} items".

As far as I understand this is not part of the system (gettext), but its bindings and in result is underspecified and differs between implementations. For example [0] uses `%{ count }` while [1] uses `{{ count }}`. If I'm mistaken here, please, point me to the spec :)

Since it is a higher level replacement, this approach likely suffers from multiple limitations. First of all, I highly doubt that there is any BiDi isolation between interpolated arguments and the string leading to a common bug when RTL text (say, arabic) contains an LTR variable (say, a latin based name of a person). Fluent resolves it by wrapping all interpolated placeables in BiDi isolation marks.

Secondly, I must assume that any internationalization, such as number formatting, date formatting, etc. is also not done from within of the resolver in gettext. That, in turn, means that it may be tricky to verify that a number is formatted using eastern arabic numerals when used in arabic translation, while formatted to western arabic when used in english translation. Fluent formats all placeables using Unicode backed intl formatters (for example in JS we use ECMA402), allowing for consistency and high quality translations where placeables get formatted together with the message.

For example, in your example, will the `You have { count } items` be translated to `لديك 5 عناصر` or `لديك ٥ عناصر`? And what will happen if instead of `count`, you'd have `name: "John"`? Will it be RTL or LTR?

[0] https://hexdocs.pm/gettext/Gettext.html#content [1] https://angular-gettext.rocketeer.be/dev-guide/api/angular-g...

Re: Fluent 1.0: a localization system for natural-sounding translations

#103

Post by chungleong »

themacguffinman wrote: post #98
chungleong wrote: post #97

Earlier quoted context omitted.

It won't help much with Chinese localization, where the persistent problem is that developers assume every language has the words "yes" and "no".

Doesn't Chinese have understandable localizations for affirmative/negative responses like "是" and "没有"? I don't see the problem.

That's roughly like putting [OK] and [Cancel] into a dialog box that asks "Do you want to save this document?". Perfectly understandable but awkward, somewhat confusing to non-tech people--in a word, unfluent. Answering such questions in Chinese (and I imagine, in Welsh) requires the verb that was used in the question.

Re: Fluent 1.0: a localization system for natural-sounding translations

#104

Post by ivan_gammel »

lewisl9029 wrote: post #24
stasm wrote: post #20

Earlier quoted context omitted.

This is a great point and something that we've seen come up very often in building UIs. The good practice which we recommend to developers at Mozilla is to avoid splitting or nesting messages, because it makes it harder for translators to see the entire translation at once. We've taken a layered approach to designing Fluent: what we're announcing today is the 1.0 of the syntax and file format specification. The imple…

I've seen a few libraries that use a similar approach of parsing strings for pseudo-elements and then matching them with React elements to avoid splitting up messages, but I've always felt a lot of resistance towards adopting something like that because it means incurring the runtime cost of parsing a string for elements when you can easily have hundreds or thousands of messages being rendered at once. (Call it a pre…

I’ve been trying to solve this problem in the resource4j library for Java, which can cache rendered strings, but in the end it’s always a memory vs performance problem. I didn’t publish any artificial benchmarks, but in couple real projects (resource4j+thymeleaf) performance impact was usually negligible.

Re: Fluent 1.0: a localization system for natural-sounding translations

#105

Post by ivan_gammel »

lewisl9029 wrote: post #18
lewisl9029 wrote: post #15

One thing that I've always struggled w.r.t i18n is having to split messages up into often times less coherent chunks in order to add things like links or tooltips or styling elements in the middle of the text, which makes it more difficult to localize messages as a holistic piece independent of the source language. For a slightly contrived example to demonstrate this, let's say you have a string like this: "Please cl…

More and more, I'm starting to think maybe plain text isn't always the best abstraction to be using for defining messages for i18n. If messages were to be defined in terms of whatever primitive you're using to build your UI (i.e. React components if you're using React, and raw html template nodes if you're working with plain html), then all of this impedance mismatch might disappear. In the React case, a component or…

It will be way easier to maintain, if the L10n resources won’t be mixed with the markup or code. Check this for example: https://github.com/resource4j/resource4j

Re: Fluent 1.0: a localization system for natural-sounding translations

#107

Post by bzbarsky »

chungleong wrote: post #96
bzbarsky wrote: post #81

Earlier quoted context omitted.

It's not "less than 100 ending in 2, 3, 4". It's "2, 3, or 4 mod 10, but not 12, 13, 14 mod 100". So for example "1002" is "few" in this sense (even though it's bigger than 100) but "14" is not, even though it's less than 100 and ends in 4.

You're right. I forgot about the teens. I don't know about Czech, but in Polish the nominative plural isn't used for numbers higher than 100. "1024 bajtów" and not "1024 bajty". If I remember the rules correctly.

That's a good point. The rules I described are for Russian, and other Slavic languages may differ. In particular, looks like Polish is pretty similar to Russian here at least in terms of where the category boundaries are, but Czech is different.

As far as the cases used go, 1024 would take the genitive singular in Russian. 1025 would take the genitive plural. 1021 would take the nominative singular. Nominative plural is not used at all when counting things in Russian.

Re: Fluent 1.0: a localization system for natural-sounding translations

#108

Post by indentit »

indentit wrote: post #106

it'd be great to see syntax highlighting for FTL files in the popular text editors, but I guess they would have to be unofficial unless a member of Mozilla/Fluent team wants to maintain it...

I see that the playground has syntax highlighting and uses `ace`, whose syntax definitions are defined in JavaScript [0] and look fairly usable. I guess it could even be converted into a `.sublime-syntax` file without too much trouble :)

[0]: https://github.com/projectfluent/play/blob/a4f49a4a7eeb93535...

Re: Fluent 1.0: a localization system for natural-sounding translations

#109

Post by zbraniecki »

indentit wrote: post #106

it'd be great to see syntax highlighting for FTL files in the popular text editors, but I guess they would have to be unofficial unless a member of Mozilla/Fluent team wants to maintain it...

I started https://github.com/projectfluent/fluent.vim but my vimscript is terrible and I will take all help I can get :)

Re: Fluent 1.0: a localization system for natural-sounding translations

#110

Post by stasm »

drdaeman wrote: post #79
stasm wrote: post #20

Earlier quoted context omitted.

This is a great point and something that we've seen come up very often in building UIs. The good practice which we recommend to developers at Mozilla is to avoid splitting or nesting messages, because it makes it harder for translators to see the entire translation at once. We've taken a layered approach to designing Fluent: what we're announcing today is the 1.0 of the syntax and file format specification. The imple…

> `[one] 1 time *[other] {$clickCount} times` How does this work for languages that have more complex pluralization rules? E.g. in Russian it's "1 раз", "2 раза", "11 раз", "12 раз", "22 раза" and "55 раз" - the case depends on the number ending, with exceptions for 11, 12, 13 and 14.

It's up to the localizer to define variants corresponding to the language's plural categories. For Russian, that's (one, few, many). Interestingly, this particular example could simplified to (few, *), because "раз" is good for both 1, 5, 11, 55, etc. See https://projectfluent.org/play/?id=7d22f87c04b23b86d9f9149d5... for an example of this in action.

Authoring tools can help here, too. Pontoon, Mozilla's translation management system, pre-populates plural variants based on the number of plural categories defined in Unicode's CLDR.

Post reply on HN