Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenBSD removes support for non-UTF8 locales

marc.info

Post reply on HN
Print view View original

121–130 of 186 posts

Re: OpenBSD removes support for non-UTF8 locales

#121

Post by Dylan16807 »

kazinator wrote: post #107
opk wrote: post #87

Earlier quoted context omitted.

My first thought was, what about the "C" locale so good to see that question already answered. I really wish there was some sort of standard "U" locale that would be the same as "C" but UTF-8, and ISO rather than US format dates.

That locale pseudo-exists. It's called "don't call the evil setlocale function, write in C90 as much as possible, do your own UTF-8 encoding and decoding, and implement the exact default date format you want with your own strftime string or whatever."

That doesn't exactly help me as a user, and possibly makes things worse as some things respect locale and some don't.

Re: OpenBSD removes support for non-UTF8 locales

#122

Post by saljam »

baudehlo wrote: post #83
peterfirefly wrote: post #52

Earlier quoted context omitted.

8601 also gives you proper week numbering (which Europeans tend to like) + weeks start on Monday, after the weekend.

In some countries the weekend isn't Saturday and Sunday though. So the start of the week is kind of arbitrary.

Saudi Arabia used to have it's weekend on Thursday & Friday. Recently they've switched to Friday & Saturday.

Re: OpenBSD removes support for non-UTF8 locales

#123

Post by rspeer »

zdw wrote: post #95
toyg wrote: post #5

Earlier quoted context omitted.

And ISO 8601. Sweet, sweet ISO 8601...

Or the even stricter RFC3339 date format, which is ISO8601 without the dumb bits: http://www.ietf.org/rfc/rfc3339.txt

Neat.

I wonder if anyone collects a list of famous standards that have been superseded by compatible, better standards. Another example would be that many people know of ISO 639 language codes, but BCP 47 is a clearer, more relevant standard.

Re: OpenBSD removes support for non-UTF8 locales

#124

Post by hyperion2010 »

RexRollman wrote: post #17
fletchowns wrote: post #4

I dream of a world where everything is UTC, UTF-8, and metric.

Personally, I wish everyone used the 24:00 clock. Maybe the military has messed me up, but I really prefer seeing something like 18:22 over 6:22pm. It just seems simplier.

I cannot tell you the number of times in my life that a 12 hour clock has lead to a _major_ fuckup. I run all my personal clocks on 24hr to prevent said fuckups, but somehow I still feel weird telling people a meeting is at 15:00 in an email.

Re: OpenBSD removes support for non-UTF8 locales

#125

Post by jerf »

protomyth wrote: post #80
gh02t wrote: post #75

Earlier quoted context omitted.

It's not arbitrary at all, it's for backwards compatibility. Previously the meter was defined in terms of the emission lines of krypton-86, which was a distance that we could measure most precisely with an interferometer. Before that it was based on the size of the earth, which could be measured with a astronomical calculations. And originally it was based on the length of a pendulum that produced a period of a 1 sec…

"It's not arbitrary at all, it's for backwards compatibility." The original definition of the meter was one ten-millionth of the distance from the equator to the North Pole. Sure, we got more precise, but its still arbitrary. The only interesting thing about metric is the relation of length, volume, mass. But, a liter is not a cubic meter, nope - its a cubic decimetre, another arbitrary decision.

The units that have the best claim to being non-arbitrary tend to be of rather inconvenient orders of magnitude: https://en.wikipedia.org/wiki/Natural_units#Systems_of_natur...

A certain amount of arbitrariness is inevitable at human scale.

Re: OpenBSD removes support for non-UTF8 locales

#126

Post by kragen »

stsp wrote: post #77
throwaway2048 wrote: post #18

Earlier quoted context omitted.

I'm willing to bet a large amount that non UTF-8 encoding were broken and nobody cared enough to bother fixing them. OpenBSD does not hesitate to nuke legacy stuff that gets broken. Which i feel is ultimately for the best, because half-assed support that barely functions is worse than no support at all many times.

It was in fact intentionally broken to find out where removing single-byte locales hurts our users most. We have a hackathon coming up with devs committed to making UTF-8 work in more base utilities. If that works out, and the most sore points of latin1/koi-8/etc users have been adequately addressed, 5.9 will ship with only the UTF-8 locale (and of course the default "C" locale -- ASCII). If this approach turns out t…

Oh, I didn't realize you weren't removing "C"! Thank you for explaining!

Re: OpenBSD removes support for non-UTF8 locales

#127

Post by kragen »

lsaferite wrote: post #92
nightpool wrote: post #71

Earlier quoted context omitted.

> he lost interest in living I get what you're trying to accomplish with the parallel construction, but that's a pretty callous way to describe it :/

He died from a massive drug overdose right?

Yes, although we didn't find out for sure until several months later. It was too massive to be an accident.

Re: OpenBSD removes support for non-UTF8 locales

#128

Post by kragen »

geofft wrote: post #64
kragen wrote: post #3

I wonder what the pros and cons weighed in the discussion were. Clearly not supporting Unicode text in non-UTF-8 locales (except through, like, some kind of compatibility function, like recode or iconv) is the Right Thing. One problem that I have is that current UTF-8 implementations typically are not "8 bit clean", in the sense that GNU and modern Unix tools typically attempt to be; they crash, usually by throwing a…

For the benefit of others (the link is nonobvious), here's Markus Kuhn's presentation of UTF-8B: http://hyperreal.org/~est/utf-8b/releases/utf-8b-20060413043... The tl;dr is to map an invalid UTF-8 byte n to code point U+DC00 + n, which puts it in the code point range reserved for the second part of a surrogate pair. (In UTF-16, a 16-bit value between D800 and DBFF followed by a 16-bit value between DC00 and DFFF is…

Agreed, except that UTF-8B also lets you round trip UTF-8B → UCS-4 → UTF-8B safely, not just via UTF-16.

Re: OpenBSD removes support for non-UTF8 locales

#129

Post by kragen »

nightpool wrote: post #71
kragen wrote: post #30

Earlier quoted context omitted.

What would be great would be if someone would take up UTF-8B again. I mentioned his death because otherwise you might think he lost interest in the project, but no, he lost interest in living.

> he lost interest in living I get what you're trying to accomplish with the parallel construction, but that's a pretty callous way to describe it :/

I'm sorry I upset you. I didn't mean to.

Re: OpenBSD removes support for non-UTF8 locales

#130

Post by vacri »

kazinator wrote: post #110
robin_reala wrote: post #97

Earlier quoted context omitted.

In the UK too traditionally, although that’s changing (by convention).

Yet, in America, people say things like "do you have plans for the weekend?" Or, "What did you do last weekend?" Nobody ever says, "Do you have plans for the upcoming two days which, respectively, constitute the end of this week and the start of the next one?" So, Americans and Brits are inconsistent. They have "the weekend" which is a block of two days when salaried people with regular working hours don't work; and…

Neither does anybody say "What are you doing for the holidays constituted of Christmas Day and selected other days around it?"
Post reply on HN