OpenBSD removes support for non-UTF8 locales
1–10 of 186 posts
Re: OpenBSD removes support for non-UTF8 locales
#2Heh, I initially read it as "improves", and was wondering why they'd bother. Removing it is surprising, but makes sense.
Re: OpenBSD removes support for non-UTF8 locales
#3I wonder what the pros and cons weighed in the discussion were.
Clearly not supporting Unicode text in non-UTF-8 locales (except through, like, some kind of compatibility function, like recode or iconv) is the Right Thing. One problem that I have is that current UTF-8 implementations typically are not "8 bit clean", in the sense that GNU and modern Unix tools typically attempt to be; they crash, usually by throwing an exception, if you feed them certain data, or worse, they silently corrupt it.
Markus Kuhn suggested "UTF-8B" as a solution to this problem some years ago. Quoting Eric Tiedemann's libutf8b blurb, "utf-8b is a mapping from byte streams to unicode codepoint streams that provides an exceptionally clean handling of garbage (i.e., non-utf-8) bytes (i.e., bytes that are not part of a utf-8 encoding) in the input stream. They are mapped to 256 different, guaranteed undefined, unicode codepoints." Eric's dead, but you can still get libutf8b from http://hyperreal.org/~est/libutf8b/.
Re: OpenBSD removes support for non-UTF8 locales
#4I dream of a world where everything is UTC, UTF-8, and metric.
Re: OpenBSD removes support for non-UTF8 locales
#5I dream of a world where everything is UTC, UTF-8, and metric.
And ISO 8601. Sweet, sweet ISO 8601...
Re: OpenBSD removes support for non-UTF8 locales
#6How does locale work on the keyboard side, then? What determines whether text entry is right to left or left to right?
Re: OpenBSD removes support for non-UTF8 locales
#7How does locale work on the keyboard side, then? What determines whether text entry is right to left or left to right?
Exactly the same as before -- your programs just expect UTF8 codepoints as input.
Re: OpenBSD removes support for non-UTF8 locales
#8I dream of a world where everything is UTC, UTF-8, and metric.
UTC is (as everyone knows) a bit problematic due to leap seconds. Different software systems handle the leap seconds somewhat differently. Handling leap seconds is is actually quite difficult if you want to get it absolutely correct. In 99% of cases the problems are just ignored (e.g. "it doesn't matter if the chart is slightly odd looking when you look at the moment of the leap second").
There's also the problem that every software using UTC must be updated at least once every six months. That may be a lesser problem these days, but is still somewhat relevant especially in various industries.
I'd probably go with TAI and just convert the dates to the "human readable" format in the UI. Of course, that's not trivial either.
Re: OpenBSD removes support for non-UTF8 locales
#9How does locale work on the keyboard side, then? What determines whether text entry is right to left or left to right?
Presumably if you press the RTL key on your keyboard, or if you enter the sequence of codes that converts it from LTR to RTL.
Re: OpenBSD removes support for non-UTF8 locales
#10I dream of a world where everything is UTC, UTF-8, and metric.
UTC is (as everyone knows) a bit problematic due to leap seconds. Different software systems handle the leap seconds somewhat differently. Handling leap seconds is is actually quite difficult if you want to get it absolutely correct. In 99% of cases the problems are just ignored (e.g. "it doesn't matter if the chart is slightly odd looking when you look at the moment of the leap second"). There's also the problem tha…
For those who don't know about TAI:
https://en.wikipedia.org/wiki/International_Atomic_Time
International Atomic Time (TAI, from the French name Temps Atomique International[1]) is a high-precision atomic coordinate time standard based on the notional passage of proper time on Earth's geoid.[2] It is the basis for Coordinated Universal Time (UTC), which is used for civil timekeeping all over the Earth's surface, and for Terrestrial Time, which is used for astronomical calculations. As of 30 June 2015 when the last leap second was added,[3] TAI is exactly 36 seconds ahead of UTC. The 36 seconds results from the initial difference of 10 seconds at the start of 1972, plus 26 leap seconds in UTC since 1972.
Time coordinates on the TAI scales are conventionally specified using traditional means of specifying days, carried over from non-uniform time standards based on the rotation of the Earth. Specifically, both Julian Dates and the Gregorian calendar are used. TAI in this form was synchronised with Universal Time at the beginning of 1958, and the two have drifted apart ever since, due to the changing motion of the Earth.