Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenBSD removes support for non-UTF8 locales

marc.info

Post reply on HN
Print view View original

101–110 of 186 posts

Re: OpenBSD removes support for non-UTF8 locales

#101

Post by dietrichepp »

protomyth wrote: post #50
fletchowns wrote: post #4

I dream of a world where everything is UTC, UTF-8, and metric.

I'd prefer to skip metric and redefine the foot as the distance light travels in a nanosecond ~ 11.8 inches. The length of the path travelled by light in vacuum during a time interval of 1/299792458 of a second seems a bit arbitrary to me. I would also vote for any candidate that would ban anything but powers of 2 in the definition of computer storage.

I don't get the fascination with powers of two in storage. RAM I understand, because you're counting address lines. But storage? What sense does that make?

I have an SD card with 16.0 GB of space available, and I'm recording video at 9.00 Mbit/s. How many hours of footage? Well, (16.0e3 MB) * (8 bit/B) / (9.00 Mbit/s) = 1.42e4 s, or 3.95 hours.

Now do the computation with binary units.

I have an SD card with 14.9 GiB of space available, and I'm recording video at 9.00 Mbit/s. How many hours of footage? Well, (14.9 GiB) * (1e-6 * 1024^3 MB / GiB) * (8 bit/B) / (9.00 Mbit/s) = ...

Why make people do extra math? Shouldn't we choose units that make things easier to calculate, not harder?

Re: OpenBSD removes support for non-UTF8 locales

#102

Post by hyperpape »

kstrauser wrote: post #86
maxerickson wrote: post #73

Earlier quoted context omitted.

The imperial choice of using highly composite numbers is at least as convenient for mental arithmetic as decimals.

What's 1/10th of 3 3/8"?

27/80. Asking HN seems like a roundabout way of getting the answer, though.

Re: OpenBSD removes support for non-UTF8 locales

#103

Post by [deleted] »

dietrichepp wrote: post #100
joshuapants wrote: post #74

Earlier quoted context omitted.

> 1. it does have the benefit that it's cross-cultural, you can actually talk to people outside your cultural bubble We already do that. > 2. it's got the advantage that you're using the same measurements as all your trade partners so people don't need two production lines anymore Two production lines? For what? > 3. it's got the advantage that people going into science and engineering don't need to build a whole new…

> Yup, failure to use the metric system in everyday life is why the US has the worst scientists and produces the least scientific output. Oh, wait. Failure to use the metric system everywhere cost NASA a $125 million Mars orbiter just 16 years ago, and yet here you are, insisting that this is not a problem, and throwing in a non sequitur to justify the position.

[deleted]

Re: OpenBSD removes support for non-UTF8 locales

#104

Post by cthalupa »

dietrichepp wrote: post #100
joshuapants wrote: post #74

Earlier quoted context omitted.

> 1. it does have the benefit that it's cross-cultural, you can actually talk to people outside your cultural bubble We already do that. > 2. it's got the advantage that you're using the same measurements as all your trade partners so people don't need two production lines anymore Two production lines? For what? > 3. it's got the advantage that people going into science and engineering don't need to build a whole new…

> Yup, failure to use the metric system in everyday life is why the US has the worst scientists and produces the least scientific output. Oh, wait. Failure to use the metric system everywhere cost NASA a $125 million Mars orbiter just 16 years ago, and yet here you are, insisting that this is not a problem, and throwing in a non sequitur to justify the position.

Science and engineering have already made the switch to metric. NASA and JPL aren't using imperial when designing their probes and rockets anymore.

Re: OpenBSD removes support for non-UTF8 locales

#105

Post by feld »

nawitus wrote: post #8
fletchowns wrote: post #4

I dream of a world where everything is UTC, UTF-8, and metric.

UTC is (as everyone knows) a bit problematic due to leap seconds. Different software systems handle the leap seconds somewhat differently. Handling leap seconds is is actually quite difficult if you want to get it absolutely correct. In 99% of cases the problems are just ignored (e.g. "it doesn't matter if the chart is slightly odd looking when you look at the moment of the leap second"). There's also the problem tha…

Easy, we just kill off the idea of leap seconds. Someone please convince Russia and UK to agree so we can do it.

Re: OpenBSD removes support for non-UTF8 locales

#106

Post by kazinator »

RexRollman wrote: post #17
fletchowns wrote: post #4

I dream of a world where everything is UTC, UTF-8, and metric.

Personally, I wish everyone used the 24:00 clock. Maybe the military has messed me up, but I really prefer seeing something like 18:22 over 6:22pm. It just seems simplier.

The real beauty of the 24:00 clock is that it never actually shows that; at most 23:59. Then it goes to 0:00.

The 12:00 clock would be more respectable if it went from 0:00 (midnight/a.m.) to 11:59 a.m. then to 0:00 (noon/p.m.) and to 11:59 p.m. and never showed 12:00. (Let alone continuously flash such a thing as a demand that the time be set.)

Re: OpenBSD removes support for non-UTF8 locales

#107

Post by kazinator »

opk wrote: post #87
stsp wrote: post #77

Earlier quoted context omitted.

It was in fact intentionally broken to find out where removing single-byte locales hurts our users most. We have a hackathon coming up with devs committed to making UTF-8 work in more base utilities. If that works out, and the most sore points of latin1/koi-8/etc users have been adequately addressed, 5.9 will ship with only the UTF-8 locale (and of course the default "C" locale -- ASCII). If this approach turns out t…

My first thought was, what about the "C" locale so good to see that question already answered. I really wish there was some sort of standard "U" locale that would be the same as "C" but UTF-8, and ISO rather than US format dates.

That locale pseudo-exists. It's called "don't call the evil setlocale function, write in C90 as much as possible, do your own UTF-8 encoding and decoding, and implement the exact default date format you want with your own strftime string or whatever."

Re: OpenBSD removes support for non-UTF8 locales

#108

Post by kazinator »

ptx wrote: post #60
kragen wrote: post #3

I wonder what the pros and cons weighed in the discussion were. Clearly not supporting Unicode text in non-UTF-8 locales (except through, like, some kind of compatibility function, like recode or iconv) is the Right Thing. One problem that I have is that current UTF-8 implementations typically are not "8 bit clean", in the sense that GNU and modern Unix tools typically attempt to be; they crash, usually by throwing a…

Kuhn's idea is also used in in Python 3, so that garbage bytes can (optionally!) be decoded to Unicode strings and later losslessly turned back into the same bytes, which ensures (e.g.) that filenames that can't be decoded can still be used: https://www.python.org/dev/peps/pep-0383/

Interesting; I implemented exactly the same thing in the TXR language. I can read an arbitrary file in /bin/ as UTF-8 to a string, and when that string is converted to UTF-8, it reproduces that file exactly. All invalid bytes go to DCXX, including the null character. The code U+DC00 is called "pnul" (pseudo-null) and can even be written like #\pnul in the language as a character constant. Thanks to pnul, you can easily manipulate data that contains nulls, like /proc//environ, or null-delimited strings from GNU "xargs -0". The underlying C strings are nicely null-terminated with the real U+0000 NUL, and everything is cool.

Re: OpenBSD removes support for non-UTF8 locales

#110

Post by kazinator »

robin_reala wrote: post #97
FullyFunctional wrote: post #91

Earlier quoted context omitted.

In the US perhaps, not in Europe

In the UK too traditionally, although that’s changing (by convention).

Yet, in America, people say things like "do you have plans for the weekend?" Or, "What did you do last weekend?"

Nobody ever says, "Do you have plans for the upcoming two days which, respectively, constitute the end of this week and the start of the next one?"

So, Americans and Brits are inconsistent. They have "the weekend" which is a block of two days when salaried people with regular working hours don't work; and they have Sunday as not the week end, but rather the beginning; or the "front end" of the next week. Which means that the two days cannot be the weekend; they are two different ends of two different weeks.

> that's changing (by convention)

It's changing because people have to confront the above reasoning and realize that a week beginning in the middle of something that they have been calling "the weekend" for decades is silly.

Post reply on HN