Live data from Hacker News

Fluent 1.0: a localization system for natural-sounding translations

hacks.mozilla.org

61–70 of 117 posts

Re: Fluent 1.0: a localization system for natural-sounding translations

#61
post #20

One thing that I've always struggled w.r.t i18n is having to split messages up into often times less coherent chunks in order to add things like links or tooltips or styling elements in the middle of the text, which makes it more difficult to localize messages as a holistic piece independent of the source language. For a slightly contrived example to demonstrate this, let's say you have a string like this: "Please cl…

This is a great point and something that we've seen come up very often in building UIs. The good practice which we recommend to developers at Mozilla is to avoid splitting or nesting messages, because it makes it harder for translators to see the entire translation at once. We've taken a layered approach to designing Fluent: what we're announcing today is the 1.0 of the syntax and file format specification. The imple…

Why fluent-react and not (in addition to) fluent-web-components? :(

Re: Fluent 1.0: a localization system for natural-sounding translations

#62
post #29

Does it handle the "x of y" in Slavic languages correctly? For example in Polish: "Page 3 of 4" is "Strona 3 z 4" "Page 3 of 100" is "Strona 3 ze 100"

This is an excellent question and a very good use-case. I'm a Polish speaker myself, so I can definitely relate. It's also a good excuse for me to talk a little bit more about the advanced features of Fluent.

The Fluent Syntax is a simple declarative DSL. By design, it doesn't allow translators to build complex conditionals or use arithmetic. There is, however, an escape hatch. The problem you described can be solved in Fluent with a little bit of one-time help from the developer of the source code, through a feature of Fluent called custom functions.

Translations in Fluent can use functions to format values or decide between variants. There exist built-in functions like NUMBER and DATETIME. They are rarely used because the Fluent runtime calls them on numeric and temporal values implicitly, but they can be helpful when localizers wish to use custom formatting options.

    weekday-today = Today is {DATETIME($today, weekday: "long")}.
See https://projectfluent.org/play/?id=a3540d4f02c104a634adbfc0e... for a live example of DATETIME.

There can also be custom functions, defined during the initialization of the runtime. In Firefox, we use one such function called PLATFORM: https://searchfox.org/mozilla-central/rev/d33d470140ce3f9426.... It can be used as follows:

    open-preferences = {PLATFORM() ->  
        [windows] Open Options
       *[other] Open Preferences
    }
The logic of custom functions is entirely up to developers and the localization needs of the UI. In https://github.com/projectfluent/fluent/issues/228#issuecomm..., for instance, I suggested using a custom function to handle negative and positive floor numbers.

A custom function can also cater to the use-case you described. A simple and possibly naive implementation in JavaScript could look like the following one:

    function NUMBER_HEAD(num) {
        while (num > 999) {
            num /= 1e3;
        }
        let first = num.toString()[0];
        return num 
I wrote this with Polish in mind, but it could be useful to other languages in which numerals are named after the first thousand-triple, in a left to right order. Depending on the exact product requirements, the function could be called NUMBER_HEAD_POLISH, or perhaps NUMBER_HEAD_TRIPLE_FIRST_DIGIT :)

Once defined, the function can be used as follows:

    # The Polish copy can take advantage of the custom function.
    page-of = {NUMBER_HEAD($pageTotal) ->
        [1xx] Strona {$pageCurrent} ze {$pageTotal}
       *[other] Strona {$pageCurrent} z {$pageTotal}
    }
This method still requires some work from developers, but it only needs to happen once and in a single palce in code: where the Fluent runtime is initialized. Because they are code, custom functions can be reviewed and tested just as any other code in the code base, to help ensure that they do what they claim to :)

Importantly, the use of the custom function is completely opt-in.

    # The English copy doesn't need any special handling.
    page-of = Page {$pageCurrent} of {$pageTotal}
All localization callsites remain unchanged, and all existing translations remain functional.

Re: Fluent 1.0: a localization system for natural-sounding translations

#63
post #10

In the Czech example, how is it obvious that `few` stands for 2, 3, 4? Is that just how the concept of "few" (note the English term) is defined and understood by all Cezch speakers and thus this language specific meaning is encoded by Mozilla to map to the range 2-4? My point is that while there might be a concept of "few" that does map uniquely to that range, I am not sure naming the keyword "few" is the right name…

(Author of the blog post here.) Great question, thanks! Unicode defines six categories of plural forms: zero, one, two, few, many, and other. The names of these categories always appear in English. Unicode also maintains a collection of all mappings of numerical rules to these categories, for all languages supported by the CLDR. See http://www.unicode.org/cldr/charts/latest/supplemental/langu... for the mapping corre…

Care to comment about how it might handle one bold word or a link mid sentence?

Re: Fluent 1.0: a localization system for natural-sounding translations

#64
post #22
post #20

Earlier quoted context omitted.

This is a great point and something that we've seen come up very often in building UIs. The good practice which we recommend to developers at Mozilla is to avoid splitting or nesting messages, because it makes it harder for translators to see the entire translation at once. We've taken a layered approach to designing Fluent: what we're announcing today is the 1.0 of the syntax and file format specification. The imple…

Would this also allow for translating e.g. the 's `title` attribute, or e.g. an `aria-label`?

What's stopping you from doing that now? Just add a new translation for those labels in the source document and bind it to the attribute tag.

Re: Fluent 1.0: a localization system for natural-sounding translations

#65
post #61
post #20

Earlier quoted context omitted.

This is a great point and something that we've seen come up very often in building UIs. The good practice which we recommend to developers at Mozilla is to avoid splitting or nesting messages, because it makes it harder for translators to see the entire translation at once. We've taken a layered approach to designing Fluent: what we're announcing today is the 1.0 of the syntax and file format specification. The imple…

Why fluent-react and not (in addition to) fluent-web-components? :(

One advantage I can imagine is that you can prerender the React components, outputting e.g. plain HTML. With tools like e.g. React Static, that means you can somewhat ergonomically generate different static websites for different languages, avoiding the runtime costs of looking up the correct strings.

Re: Fluent 1.0: a localization system for natural-sounding translations

#66
post #53
post #48

Earlier quoted context omitted.

Wouldn’t you just wrap that word in another selector?

Will the system still be sanely usable for translators?

Yeah this is where these things fall apart. I haven't used Mozilla's Fluent but I used a very similar closed source system at another company some years back. Some failure modes:

- Gender agreement is not trivial. In French, in "Mary bought it" the verb needs to match the gender and number of the object not the subject: "Marie l'a acheté" vs "Marie l'a achetée" vs "Marie les a achetés" depending on the gender/number of the "it" object. But in most other cases the verb needs to match the subject in gender and number, in Polish "Maria kupila" vs "Stas kupil" vs "Oni kupili".

- In many languages nouns need to agree in case, gender, and number with the phrase they're in, even in English we see this with pronouns: "this is he" vs "this is his".

- And not to mention number agreement between pronoun and either subject or object, depending on context: "this is the button" vs "these are the buttons" - but also "this hovers over the button" vs "these hover over the button" etc. Pronouns in general are a world of hurt, as are copulae (is/are/etc)

- So once we want more complex sentences, simple word tagging like $gender becomes insufficient, because now there's multi-party agreement to worry about, we have to worry about $gender_subject and $object_gender_number_case, etc.

This becomes completely untenable for all but the most technical translators. Maybe those are easy to find for a world famous project like Firefox. Unfortunately, not so for a run of the mill commercial project.

Re: Fluent 1.0: a localization system for natural-sounding translations

#67
post #41

Re: genedered pluralized example in the article. How will the system deal with the fact, that in some languages (Czech, too): ($count -> Jana added {n} {apples|apple}) ($gender -> to {his|her} profile) the $gender will affect what form the word "added" should take. You're suddenly dealing with possibilities($count)*possibilities($gender) variants of the sentence

It's possible to build this message in Fluent with nested selectors, or with adjacent selectors. I built an example using Polish, since that's a language I know best. (To be 100% correct in Polish, I'd need to use a different possessive pronoun, but doing so would actually remove the double use of gender from your question.)

https://projectfluent.org/play/?id=2d7ab4b7ed1c4d9656475614f...

It's a complex piece of UI and consequently, the resulting Fluent message is also quite complex. But possible to build :)

Re: Fluent 1.0: a localization system for natural-sounding translations

#68
post #53
post #48

Earlier quoted context omitted.

Wouldn’t you just wrap that word in another selector?

Will the system still be sanely usable for translators?

It's certainly complicated but I think there is no way around it if you want to have correct translations. The same kind of system is used for translating MediaWiki and it seems to work great there. Example message: https://translatewiki.net/w/i.php?title=MediaWiki:Logentry-b...

Re: Fluent 1.0: a localization system for natural-sounding translations

#69
My take on how to solve the natural-sounding translation problem: https://www.lokalized.com/#a-more-complex-example

The magic is a tiny expression language which understands plural cardinalities, ordinals, etc. so a translator can encode all required logic in a JSON file - the application code can be "dumb".

Re: Fluent 1.0: a localization system for natural-sounding translations

#70
post #47

Earlier quoted context omitted.

As far as I can tell Fluent has no builtin support for tagging a phonemic information to messages or arguments (it only supports plural rules and number formatting largely derived from the CLDR). You can probably specify special cases with selectors (but it will quickly go absurd, for Polish I guess that applies to 6, 7, 16, 17, 60..79, 100..199, 600..699 and so on?) or have an external function. Context: Polish prep…

Funnily enough "Strona 3 z 6" and "Strona 3 z 7" sounds correct but "Strona 3 z 100" doesn't. So I think it's only words starting with "s", not "sz" nor "si". And only 0 starts with "z" so it's not a problem (you never have "X out of 0"). So the only special case is for 100-199, 100 000-199 999, etc.

A filler vowel is needed with certain consonant clusters. 100 is "stu" and "st" requires it, whereas "dwustu" (200), "trzystu" (300), "czterystu" (400) don't. I don't remember the list. "mn" is another combo that needs the vowel: "ze mnóstwa".
Post reply on HN