Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The plague of emoji insertion in French docs

bibelo.info

Post reply on HN
Print view View original

161–170 of 211 posts

Re: The plague of emoji insertion in French docs

#161

Post by cryptonector »

gnubison wrote: post #160
cryptonector wrote: post #145

Earlier quoted context omitted.

It's more complicated than you imagine it is. Basically you need to know both: whether the current locale is a French locale, and also whether original text was written in a French locale. The former is easy enough, but also very annoying to multilingual people since one might run in a Spanish locale but occasionally write in French. So that's not a solution. The latter is... hard to do, because while Unicode has lan…

> Unicode has language tags that you can embed in documents, those are deprecated and they were never well supported, Huh? MDN doesn’t mention this … why are they deprecated?

Yes, [sadly] deprecated by the Unicode Consortium in Unicode 5.2, and by the IETF in RFC 6082. Nor have they be undeprecated since them (see https://www.unicode.org/versions/Unicode15.0.0/ch05.pdf#G115...).

IMO we should have tried harder to make Unicode language tags useful and used. But it didn't happen, so they're a thing of the past. Of course, they're still there, and one could attempt to resurrect them, but most likely one would fail.

Choice quotes below:

https://www.rfc-editor.org/rfc/rfc6082

  > RFC 2482, "Language Tagging in Unicode Plain
  > Text" [RFC2482], describes a mechanism
  > for using special Unicode language tag
  > characters to identify languages when needed.
  > It is an idea whose time never quite came.
  > It has been superseded by whole-transaction
  > language identification such as the MIME
  > Content-language header [RFC3282] and more
  > general markup mechanisms such as those
  > provided by XML.  The Unicode Consortium
  > has deprecated the language tag character
  > facility and strongly recommends against
  > its use.  RFC 2482 has been moved to
  > Historic status to reduce the possibility
  > that Internet implementers would consider
  > that tagging system an appropriate mechanism
  > for identifying languages.
  >
  > A discussion of the status of the language tag
  > characters and their applicability appears
  > in Section 16.9 of The Unicode Standard
  > [Unicode52].
https://www.unicode.org/versions/Unicode5.2.0/ch16.pdf (section 9 of that chapter, 16)

  > 16.9 Deprecated Tag Characters 519 The Unicode
  > Standard, Version 5.2 Copyright © 1991–2009
  > Unicode, Inc.  for detailed recommendations
  > on the use of U+FFFD as replacement for
  > ill-formed sequences. See also Section 5.3,
  > Unknown and Missing Characters for related
  > topics.  16.9 Deprecated Tag Characters
  > Deprecated Tag Characters: U+E0000–U+E007F
  > The characters in this block provide a
  > mechanism for language tagging in Unicode
  > plain text. These characters are deprecated,
  > and should not be used—particularly with any
  > protocols that provide alternate means of
  > language tagging. The Unicode Standard recom-
  > mends the use of higher-level protocols, such as
  > HTML or XML, which provide for language tagging
  > via markup. See Unicode Technical Report #20,
  > “Unicode in XML and Other Markup Languages.”
  > The requirement for language information embedded
  > in plain text data is often overstated, and
  > markup or other rich text mechanisms constitute
  > best current practice. See Section 5.10,
  > Language Information in Plain Text for further
  > discussion.
(Reformatting is mine.)

Re: The plague of emoji insertion in French docs

#162

Post by Am4TIfIsER0ppos »

karmakaze wrote: post #25

I didn't realize French was written with spaces before colons. It might be time to lose that space, similar to how we don't double space after period. I'm not saying G.Docs or any editor should dictate how French is written, only being pragmatic if I have to choose between (no space): or unwanted emojis ending up in docs.

> similar to how we don't double space after period

What do you mean? Every sentence in this comment has two spaces after the punctuation. That one and this one. Just because browsers and html condense it all into one space doesn't make it right.

Re: The plague of emoji insertion in French docs

#163

Post by [deleted] »

cryptonector wrote: post #147
Aissen wrote: post #42

Typographic nerd: technically, you should you use a non-breaking space (espace insécable) before colons in french. Doing that does not show the popup. Wait, what do you mean your keyboard layout does not have it? Keyboard layout nerd: you should use a keymap that has it. Like AFNOR's latest AZERTY or BÉPO. Historical nerd: traditionally wordprocessors have auto-inserted non-breaking spaces before colons, why should I…

> Historical nerd: traditionally wordprocessors have auto-inserted non-breaking spaces before colons, why should I care now ? Because I don't want to switch locales in order to write short bits of text in another language. I don't even want to have to switch keyboard layouts -- just use compose sequences for diacritical marks and so on. And I don't use a French locale, but I do write French text sometimes. I would im…

[deleted]

Re: The plague of emoji insertion in French docs

#164

Post by sdeframond »

makapuf wrote: post #103
sdeframond wrote: post #68

Earlier quoted context omitted.

Or, word processors could understand that the pattern "some-chars" + + ":" must be treated as a single word in French. (I guess it's more complicated than I imagine it is, alright)

And thus impossible to use the emoji inserting feature, and other bugs (silently replacing whitespace with different but looking the same whitespace will make for fun issues)

MS Word actually replaces normal whitespaces with non breaking spaces in some cases. Kinda works.

Re: The plague of emoji insertion in French docs

#165

Post by Kwpolska »

cryptonector wrote: post #142
jobigoud wrote: post #101

Earlier quoted context omitted.

A modern solution is simply to have non-breaking space easily accessible in your keyboard layout for when you need it. In the BÉPO layout this is at SHIFT + space. Especially simple since all double punctuations (:?!; those that require an nbsp before them) are also accessed with SHIFT.

A modern solution is to abandon old, no-longer-relevant typographic language rules, or to make typographic language rules context-specific. But I agree that we need to make several alternative space characters easy to type: - non-breaking space (for this French rule) - wide space (for disambiguating sentence ending periods from non-sentence-ending periods) - zero-width non-breaking space (for preventing word-splittin…

What makes the French rules old, no-longer-relevant, invalid? Why should we change language to appease lazy software developers?

Re: The plague of emoji insertion in French docs

#166

Post by rpadovani »

Freak_NL wrote: post #41
rpadovani wrote: post #33

Earlier quoted context omitted.

Personally, I don't. I live in Muenchen, not München. When strictly needed, CTRL+C/CTRL+V from a Google search makes the trick. When writing email/docs, autocorrector for the win. Same thing in Italian: `e instead of è. Although, the German replacement is cleaner, I'd say.

> I live in Muenchen, not München. That's… quite certainly not an opinion shared with all of your fellow inhabitants.

You are right, but this is the internet, and the official website for the city is https://www.muenchen.de/, so ¯\_(ツ)_/¯

Re: The plague of emoji insertion in French docs

#167

Post by shakow »

cryptonector wrote: post #141
rodelrod wrote: post #44

This space before the colon and other marks in French should actually be a narrow non-breaking space (U+202F) [0]. There's no key for it in the AZERTY layout. This has been a problem since the typewriter age. People having to get on with their jobs coped with it by using a full, breaking em-space. Unless this gets replaced automatically by the word processor, you get horrid typography and misplaced line breaks all ov…

If you fail to put a space before a colon when writing in French, what happens next? Do people point and laugh? You get disciplined? Or would French speakers accept this as a better way to use the colon character? It looks like this rule is based on old typographic considerations. Much like the Spanish Royal Academy's rule that capitalized letters carry no accents (unlike the opposite French rule that capitalized let…

I don't know, what would happen in English if you didn't capitalize the days of the week? Do people point and laugh? You get disciplined? Or would English speakers accept this as a better way to write the days of the week?

No human language on Earth is in a position where it can laughs at others for their idiomatisms.

Re: The plague of emoji insertion in French docs

#168

Post by cryptonector »

Kwpolska wrote: post #165
cryptonector wrote: post #142

Earlier quoted context omitted.

A modern solution is to abandon old, no-longer-relevant typographic language rules, or to make typographic language rules context-specific. But I agree that we need to make several alternative space characters easy to type: - non-breaking space (for this French rule) - wide space (for disambiguating sentence ending periods from non-sentence-ending periods) - zero-width non-breaking space (for preventing word-splittin…

What makes the French rules old, no-longer-relevant, invalid? Why should we change language to appease lazy software developers?

Not "rules", just this rule. It has to do with typographic considerations that apply to old typesetting technologies that are no longer in use.

> Why should we change language to appease lazy software developers?

It's been done.

For example, in Spanish it is no longer the rule that "ch" and "ll" sort as if they were distinct letters (this change was made in 2010) precisely because that was such a difficult rule to implement. And that was a 256 year-old rule per-wikipedia:

  The digraphs "ch" and "ll" were
  considered single letters of the
  alphabet from 1754 to 2010 (and
  sorted separately from "c" and "l"
  from 1803 to 1994).
For another example, in Spanish capital letters were required to not carry accents, but now they are allowed to not carry accents. This was due to the use of overstriking on typewriters working to accent lower-case letters but not upper-case letters (apostrophe would collide with the glyphs for upper-case vowels). But the technology to resolve this has existed in the Spanish-speaking world for a long time now, so the rule was finally dropped. (Not accenting upper-case letter can lead to ambiguities that are annoying.)

It's not just precedent. It's that the original reason for some typographic (not even orthographic) rule is simply not relevant in 2022.

And it's not unreasonable for either French people, non-French French speakers, or just non-French French-non-users to propose the ditching of hard-to-implement French rules. Now, this particular rule is decidedly not difficult to implement, but it is an annoying rule to apply as a user -- I should know, since I speak and write French (though I am not French).

Also, it doesn't matter what the French Academy says, or what the Spanish Royal Academy says, or what Webster's dictionary says, or whatever. Language evolves, even to their consternation. Moreover, developers don't have to care that much -- I18N/G11N is fun enough, and employers have to care for legal reasons, but rules like the Spanish ch/ll rule can be much too hard even for non-lazy developers, and the Royal Spanish Academy can and did have to change, and it was for the better.

Re: The plague of emoji insertion in French docs

#169

Post by wewtyflakes »

shakow wrote: post #167
cryptonector wrote: post #141

Earlier quoted context omitted.

If you fail to put a space before a colon when writing in French, what happens next? Do people point and laugh? You get disciplined? Or would French speakers accept this as a better way to use the colon character? It looks like this rule is based on old typographic considerations. Much like the Spanish Royal Academy's rule that capitalized letters carry no accents (unlike the opposite French rule that capitalized let…

I don't know, what would happen in English if you didn't capitalize the days of the week? Do people point and laugh? You get disciplined? Or would English speakers accept this as a better way to write the days of the week? No human language on Earth is in a position where it can laughs at others for their idiomatisms.

I do not think that was their point (to laugh at other languages), but rather, that there may be contemporary situations in which warrant rethinking the idioms of a language. As an aside, I do not think many people who speak english would even notice a lack of capitalization for the days of the week.

Re: The plague of emoji insertion in French docs

#170

Post by cryptonector »

shakow wrote: post #167
cryptonector wrote: post #141

Earlier quoted context omitted.

If you fail to put a space before a colon when writing in French, what happens next? Do people point and laugh? You get disciplined? Or would French speakers accept this as a better way to use the colon character? It looks like this rule is based on old typographic considerations. Much like the Spanish Royal Academy's rule that capitalized letters carry no accents (unlike the opposite French rule that capitalized let…

I don't know, what would happen in English if you didn't capitalize the days of the week? Do people point and laugh? You get disciplined? Or would English speakers accept this as a better way to write the days of the week? No human language on Earth is in a position where it can laughs at others for their idiomatisms.

In English most people wouldn't care. You're going to get in more trouble in French schools for this than in U.S. schools (I don't know about UK schools, or elsewhere in the Anglosphere). My impression is that European culture is a lot more sensitive to these things than U.S. culture.
Post reply on HN