Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenBSD removes support for non-UTF8 locales

marc.info

Post reply on HN
Print view View original

31–40 of 186 posts

Re: OpenBSD removes support for non-UTF8 locales

#31

Post by jaawn »

adamrt wrote: post #19
ohitsdom wrote: post #14

Earlier quoted context omitted.

I'd strongly consider voting for a US presidential candidate solely on if moving to metric was a big part of their platform. So much time is wasted in school on the confusing mess that is the imperial system.

The amount of time and money to shift at this point would be astronomical wouldn't it? It would take a few generations as we would continue to have to teach imperial so people could convert when they ran across any existing media referring to the imperial system measurements. Hundred years of machines that use imperial, speedometers, odometers, books, tools, software, air conditioning, heating, manufacturing, constru…

I think it might not be as bad as we imagine. All speedometers I've seen show both miles-per-hour and kilometers-per-hour, all thermostats I've seen have an option for switching to Celsius, all scales I've owned can display both pounds and kilograms (my current one even has an option for stones), all measuring cups I've seen in a long time have both systems displayed, etc. Even lumber wouldn't necessarily be a huge issue. After all, the "2 by 4" is actually 1.5" by 3.5", which equates to 38mm by 89mm. Would it be a huge deal to round that to 40mm by 90mm?

And, in any case, most people have a mobile phone now which can quickly do unit conversions. I really think it actually is feasible, if you could get the public behind it (good luck).

Re: OpenBSD removes support for non-UTF8 locales

#32

Post by ams6110 »

ohitsdom wrote: post #14
fletchowns wrote: post #4

I dream of a world where everything is UTC, UTF-8, and metric.

I'd strongly consider voting for a US presidential candidate solely on if moving to metric was a big part of their platform. So much time is wasted in school on the confusing mess that is the imperial system.

There's really no benefit in switching for "everyday" things like temperature, speed limits, weight... it would be a huge expense that doesn't really have any payoff.

Where metric units are important, e.g. science and engineering, they are already used.

Re: OpenBSD removes support for non-UTF8 locales

#33

Post by carey »

alricb wrote: post #28
adamrt wrote: post #19

Earlier quoted context omitted.

The amount of time and money to shift at this point would be astronomical wouldn't it? It would take a few generations as we would continue to have to teach imperial so people could convert when they ran across any existing media referring to the imperial system measurements. Hundred years of machines that use imperial, speedometers, odometers, books, tools, software, air conditioning, heating, manufacturing, constru…

It's really not that bad. Canada did it in the 1970s and it's still around, and we didn't revert to the stone age or anything. SI units are already in wide use in the US, for instance for electricity (Amperes, Watts, Volts). For the most part, international trade has pushed US industries to adopt metric anyway: metric fasteners are used all over, in cars for instance. Also, countries that have switched still use inch…

My favourite is car tyre sizes. 215/60R16 means the maximum width is 215 millimetres, the aspect ratio of the cross section is 60 percent, R is both a separator and a code for radials, and it's made for 16 inch rims.

Re: OpenBSD removes support for non-UTF8 locales

#34

Post by ohitsdom »

fletchowns wrote: post #22
adamrt wrote: post #19

Earlier quoted context omitted.

The amount of time and money to shift at this point would be astronomical wouldn't it? It would take a few generations as we would continue to have to teach imperial so people could convert when they ran across any existing media referring to the imperial system measurements. Hundred years of machines that use imperial, speedometers, odometers, books, tools, software, air conditioning, heating, manufacturing, constru…

The total cost to switch has gotta be in the hundreds of billions of dollars. Has anybody done such a calculation?

Thing is, we use both systems now. What huge cost is involved by saying "Starting in 2020, the metric system will be the official unit of measurement in the US." Manufacturing companies can switch as soon as they want to. Most consumers' toolsets already have both systems (wrenches, sockets, etc).

Re: OpenBSD removes support for non-UTF8 locales

#35

Post by kragen »

throwaway2048 wrote: post #18
kragen wrote: post #3

I wonder what the pros and cons weighed in the discussion were. Clearly not supporting Unicode text in non-UTF-8 locales (except through, like, some kind of compatibility function, like recode or iconv) is the Right Thing. One problem that I have is that current UTF-8 implementations typically are not "8 bit clean", in the sense that GNU and modern Unix tools typically attempt to be; they crash, usually by throwing a…

I'm willing to bet a large amount that non UTF-8 encoding were broken and nobody cared enough to bother fixing them. OpenBSD does not hesitate to nuke legacy stuff that gets broken. Which i feel is ultimately for the best, because half-assed support that barely functions is worse than no support at all many times.

If I had to guess, using my mental model of OpenBSD:

(a) most non-UTF-8-or-UTF-16 locales will choke (crash or corrupt data) in the rare case that they try to encode text outside their encoding range (the mirror image of the problem UTF-8B fixes in UTF-8);

(b) codecs have to be fast and handle untrusted strings of somewhat unpredictable lengths, making them a likely source of security holes;

(c) possible subtle bugs in a codec enable "cloaking attacks" where different parts of a system parse the same string differently; these have existed in the past with UTF-8, but would have to be rooted out of every codec;

(d) encoding text with one codec and decoding it with another also corrupts it.

So there are lots of good reasons to require the system to default to UTF-8 and use other codecs only in special cases involving backwards compatibility.

I hope you can still get reasonable performance and sensible ordering by setting LC_COLLATE=C.

Re: OpenBSD removes support for non-UTF8 locales

#36

Post by dalke »

adamrt wrote: post #19
ohitsdom wrote: post #14

Earlier quoted context omitted.

I'd strongly consider voting for a US presidential candidate solely on if moving to metric was a big part of their platform. So much time is wasted in school on the confusing mess that is the imperial system.

The amount of time and money to shift at this point would be astronomical wouldn't it? It would take a few generations as we would continue to have to teach imperial so people could convert when they ran across any existing media referring to the imperial system measurements. Hundred years of machines that use imperial, speedometers, odometers, books, tools, software, air conditioning, heating, manufacturing, constru…

We were almost there in the 1970s.

As for "a few generations", yes, certainly. We have only to look to the mishmash of unit use in the UK.

But what's wrong with taking a few generations to get there?

Turning it around, we got rid of a lot of specialized units - hogshead, chain, furlong, peck, etc. (If Peter Piper picked a peck of pickled peppers, how many many more are needed to make a bushel?) So it's clearly possible.

Also, metric countries still use non-metric terms for some cases, eg, "inches in a screen", the market price of a "barrel" of oil, and food energy in "[food] calories". This tells me that the transition can occur piecemeal.

Re: OpenBSD removes support for non-UTF8 locales

#37

Post by fletchowns »

jaawn wrote: post #31
adamrt wrote: post #19

Earlier quoted context omitted.

The amount of time and money to shift at this point would be astronomical wouldn't it? It would take a few generations as we would continue to have to teach imperial so people could convert when they ran across any existing media referring to the imperial system measurements. Hundred years of machines that use imperial, speedometers, odometers, books, tools, software, air conditioning, heating, manufacturing, constru…

I think it might not be as bad as we imagine. All speedometers I've seen show both miles-per-hour and kilometers-per-hour, all thermostats I've seen have an option for switching to Celsius, all scales I've owned can display both pounds and kilograms (my current one even has an option for stones), all measuring cups I've seen in a long time have both systems displayed, etc. Even lumber wouldn't necessarily be a huge i…

You're not thinking about all the designs, tooling, manufacturing facilities, etc to build all that stuff though. That's where the real cost is.

> Would it be a huge deal to round that to 40mm by 90mm?

It actually would be a big deal to round things like that I think. Whole designs would need to be updated to take into account the new dimensions of things.

Re: OpenBSD removes support for non-UTF8 locales

#38

Post by masklinn »

ams6110 wrote: post #32
ohitsdom wrote: post #14

Earlier quoted context omitted.

I'd strongly consider voting for a US presidential candidate solely on if moving to metric was a big part of their platform. So much time is wasted in school on the confusing mess that is the imperial system.

There's really no benefit in switching for "everyday" things like temperature, speed limits, weight... it would be a huge expense that doesn't really have any payoff. Where metric units are important, e.g. science and engineering, they are already used.

1. it does have the benefit that it's cross-cultural, you can actually talk to people outside your cultural bubble

2. it's got the advantage that you're using the same measurements as all your trade partners so people don't need two production lines anymore

3. it's got the advantage that people going into science and engineering don't need to build a whole new set of unit references because they've got the one which already works

4. the "huge expense" is pretty much made out of whole cloth for the purpose of saying you can't switch, the UK's metrication cost basically nothing except for road sign replacements which is why those are still imperial

Re: OpenBSD removes support for non-UTF8 locales

#39

Post by masklinn »

kragen wrote: post #3

I wonder what the pros and cons weighed in the discussion were. Clearly not supporting Unicode text in non-UTF-8 locales (except through, like, some kind of compatibility function, like recode or iconv) is the Right Thing. One problem that I have is that current UTF-8 implementations typically are not "8 bit clean", in the sense that GNU and modern Unix tools typically attempt to be; they crash, usually by throwing a…

> One problem that I have is that current UTF-8 implementations typically are not "8 bit clean", in the sense that GNU and modern Unix tools typically attempt to be; they crash, usually by throwing an exception

Crashing on invalid data sounds like a great idea. Leaving garbage through doesn't.

Re: OpenBSD removes support for non-UTF8 locales

#40

Post by maxerickson »

jaawn wrote: post #31
adamrt wrote: post #19

Earlier quoted context omitted.

The amount of time and money to shift at this point would be astronomical wouldn't it? It would take a few generations as we would continue to have to teach imperial so people could convert when they ran across any existing media referring to the imperial system measurements. Hundred years of machines that use imperial, speedometers, odometers, books, tools, software, air conditioning, heating, manufacturing, constru…

I think it might not be as bad as we imagine. All speedometers I've seen show both miles-per-hour and kilometers-per-hour, all thermostats I've seen have an option for switching to Celsius, all scales I've owned can display both pounds and kilograms (my current one even has an option for stones), all measuring cups I've seen in a long time have both systems displayed, etc. Even lumber wouldn't necessarily be a huge i…

If you changed the sizes of lumber, I'm pretty sure that someone would come out with "new inch" tapes that were labeled with inches and feet but accounted for the change in size (spreading the extra millimeters out across the distance).

The dimensions in use are something that people are very used to working with and they know how to do the necessary mental arithmetic to work with them.

Post reply on HN