Xterm(1) now UTF-8 by default on OpenBSD • Hacker News Board

Re: Xterm(1) now UTF-8 by default on OpenBSD

#101

Post by notalaser » Wed, Mar 09, 2016, 2:22 PM UTC

digi_owl wrote: post #94

notalaser wrote: post #83

Earlier quoted context omitted.

I don't know if it ever was a "kernel up" development process. Gnome (and KDE) both had their own, pretty complex stack, even before Freedesktop.org came up (e.g. KDE had DCOP and a bunch of other services). And they weren't exactly bug-free, either -- but at least they covered a lot less surface. I don't think it's a simple problem, and I don't think all the blame should be laid on Freedesktop.org, where a lot of go…

Not in a planned sense. But until Freedesktop, i had the impression that you had the kernel, then CLI user space, and the X and the DEs that wrapped the CLI tools in certain ways and the CLI tools in turn talked to the kernel. Thus you could go from bare kernel, to CLI to GUI in a layered manner (and fall back when a higher layer had issues). With Dbus etc the CLI has been sidelined. Now you have a bunch of daemons t…

I wouldn't have a problem with a bunch of daemons talking D-Bus to each other, if their interfaces were properly documented, if D-Bus were properly documented and, in general, if D-Bus weren't quite a mess. I mean, if it weren't for this obscurity, issuing commands over D-Bus wouldn't be that vastly different than issuing them on a console, only more verbose.

Unfortunately, there is a lot of weird interaction between all these processes. It's often badly (or not at all) documented, and what plugs where is extremely unclear. It's very messy, and it doesn't stand still long enough for someone to fix it. They just pile half-done stuff over more half-done stuff.

It's really unfortunate because the Linux kernel is, pragmatically, probably the best there is. It may not excel in specific niches (e.g. security), but overall, it does a lot of things better than, or at least about as well as BSDs, on systems where not even NetBSD boots.

Re: Xterm(1) now UTF-8 by default on OpenBSD

#102

Post by wtbob » Wed, Mar 09, 2016, 2:36 PM UTC

spiralpolitik wrote: post #90

plugnburn wrote: post #60

Earlier quoted context omitted.

Here in ex-USSR we have those problems too. Why not standardize decimal separators altogether worldwide? We're not dealing with feeking paper handwriting! If a number is printed on a computer display, it must look like 123456.78, not like "123 456,78"! Same goes for datetime representation. This localization BS has spawned an entire race of nonsense, where, for example, CSV files are not actually CSV in some regions,…

Given that the world hasn't yet agreed on if a line ends by carriage return or carriage return-line feed I would not hold out much hope on this front (although with the death of just line feed some progress on this front as been made). See also paper sizes and electrical power outlets.

> Given that the world hasn't yet agreed on if a line ends by carriage return or carriage return-line feed I would not hold out much hope on this front (although with the death of just line feed some progress on this front as been made).

Your point's correct, but linefeed hasn't died: it's still the line-ending on Unixes. Old Macs used carriage return; Windows use carriage return line feed; Unix uses linefeed. I don't know what Mac OS X uses, because I stopped using Macs before it came out.

Re: Xterm(1) now UTF-8 by default on OpenBSD

#103

Post by kazinator » Wed, Mar 09, 2016, 2:40 PM UTC

pilif wrote: post #39

cokernel_hacker wrote: post #28

Earlier quoted context omitted.

I don't think Rob meant stability. Rob was probably referring to the reality that modern Linux hasn't innovated itself past SVR4 by any appreciable amount. We are still using X, still using terminals powered by control codes, etc. Rob probably sees things like LANG and LC_ALL as bugs. His fix was UTF-8 everywhere, always. Where is Linux? Still in bag-of-bytes-o-rama.

>Rob probably sees things like LANG and LC_ALL as bugs. His fix was UTF-8 everywhere, always The problems solved by LANG or LC_ALL are not solved by UTF8 alone. Even if you use UTF8 for all your input and output, there is still the question of how to format numbers and dates to the user and how to collate strings. These things are dependent on country and language, sometimes even varying between different places in a…

LC_* is 1980's ISO C design that is unaware of things like, oh, threads. What if I want one thread to collate strings one way, and another to do it another way? Could easily happen: e.g. concurrent server fielding requests from clients in different countries.

Also, how on earth is it a good idea to make the core string routines in the library be influenced by this cruft? What if I have some locale set up, but I want part of my program to just have the good old non-localized strcmp?

The C localization stuff is founded on wrong assumptions such as: programs can be written ignorant of locale and then just localized magically by externally manipulating the behavior of character-handling library routines.

Even if that is true of some programs, it's only a transitional assumption. The hacks you develop for the sake of supporting a transition to locale-aware programming become obsolete once people write programs for localization from the start, yet they live on because they have been enshrined in standards.

Re: Xterm(1) now UTF-8 by default on OpenBSD

#104

Post by wtbob » Wed, Mar 09, 2016, 2:46 PM UTC

igravious wrote: post #2

I've been trying to teach myself some unicode code points because I'm getting sick and tired of continually Googling them and copying and pasting the result or bringing up a symbol character table. In fact, I'd say keyboards are woefully out to date. Specifically, I keep looking up † dagger (U+2020) and ‡ double-dagger (U+2021) for footnotes, black heart (U+2065) to be romantic, black star (U+2605) to talk about Davi…

XCompose is your friend: black hear suit is Compose-[1] https://github.com/cofi/dotfiles/blob/master/XCompose

Re: Xterm(1) now UTF-8 by default on OpenBSD

#105

Post by kazinator » Wed, Mar 09, 2016, 2:47 PM UTC

dijit wrote: post #54

2trill2spill wrote: post #40

Earlier quoted context omitted.

> The maintainers of the free BSD's have been carrying that torch, but they don't believe in features. Stepping into a BSD variant is like a trip back to the 1980's. Not exactly the real 1980's, but a parallel 1980's in which Unix is more robust---but the features are all rolled back, so it's just about as unpleasant to use. Sorry but that's all BS. FreeBSD is definitely a modern system, has lot's of features and is…

He wants; Games, SystemD, complicated control interfaces, and most of all a fully interactive desktop environment which mounts his drives for him and plays flash videos.. .. Without taking much time to configure it. This is fair enough but it's not what I want in a machine, openBSD might be "behind" but it feels complete, supported, sustainable and most of all "very well thought out". FreeBSD is also exceedingly good…

I do not want any of that stuff.

I found FreeBSD to be unusable simply in the command line environment. I was using only a text console login. I simply wanted a decent shell and editor.

Heck, FreeBSD wouldn't even scroll back with the de facto standard Shift-PgUp.

> mounts his drives for him

That amazing advancement in Unix usability can be achieved by something called the "automount daemon" which was introduced in the late 1980's in SunOS (the real, pre-Solaris one).

Tom Lyon developed the original automount software at Sun Microsystems: SunOS 4.0 made automounting available in 1988. [https://en.wikipedia.org/wiki/Automounter]

You basically just wrote a comment which paints a 1988 commercial Unix feature as a Linux frill that BSD people don't need.

FreeBSD has caved in and has autofs as of 10.1: https://www.freebsd.org/cgi/man.cgi?query=autofs&sektion=5

That was released in November 2014, only some 26 years after Sun rolled out the feature. Better late than never, I suppose.

Re: Xterm(1) now UTF-8 by default on OpenBSD

#106

Post by kazinator » Wed, Mar 09, 2016, 2:55 PM UTC

setra wrote: post #36

cplease wrote: post #35

Earlier quoted context omitted.

Just replying to point out that the grayed out comment above is the correct one. There is ugliness in coreutils, but it mature, functional, proven ugliness. A lot of it is even there for a reason. It's not difficult to make an elegant toy in isolation.

checkout the different implementations of echo across various operating systems: " https://gist.github.com/dchest/1091803" .

As a result, the specification for echo is a ridiculous mess which deviates from the other POSIX utilities. It doesn't have the normal command argument handling, and also has implementation-defined behaviors. It's a good command to avoid in serious scripting, other than for outputting fixed, alphanumeric strings.

That said, it's useful to have features like C escape sequences for control characters or arbitrary characters. That feature should be in the shell language. Someone needed it and hacked it into some version of echo. Others elsewhere didn't and so now it's implementation defined whether or not you get backslash processing.

Re: Xterm(1) now UTF-8 by default on OpenBSD

#107

Post by kazinator » Wed, Mar 09, 2016, 2:57 PM UTC

philh wrote: post #93

plugnburn wrote: post #60

Earlier quoted context omitted.

Here in ex-USSR we have those problems too. Why not standardize decimal separators altogether worldwide? We're not dealing with feeking paper handwriting! If a number is printed on a computer display, it must look like 123456.78, not like "123 456,78"! Same goes for datetime representation. This localization BS has spawned an entire race of nonsense, where, for example, CSV files are not actually CSV in some regions,…

> If a number is printed on a computer display, it must look like 123456.78, not like "123 456,78"! Humans find thousands separators useful. You're asking humans to give up useful things because they're hard to program. That said, I idly wonder whether they could be implemented with font kerning. The bytes could be 123456.78, but the font could render it with extra space, as 123 456.78. I don't know if it's possible…

Humans should agree on the decimal and separator symbols, the same way that they agreed on the Indo-Arabic numerals, and symbols like + (plus) and - (minus).

Re: Xterm(1) now UTF-8 by default on OpenBSD

#108

Post by kazinator » Wed, Mar 09, 2016, 3:17 PM UTC

gjvc wrote: post #59

kazinator wrote: post #19

Great! Now just drop the embarrassing man(1) page reference, and you can call it modernized. Wow, I'm surprised that the people whose buttons this pushes are able to make(1) a HN account, let alone have enough points to downvote. Think about it. There is only one man page for xterm. I fyou type "man xterm" with no section number you get that man page. If there existed an xterm(7) page, you'd still get the xterm(1) ma…

>Think about it. There is only one man page for xterm. I fyou type "man xterm" with no section number you get that man page. If there existed an xterm(7) page, you'd still get the xterm(1) man page by default. So why the hell write the (1) notation every time you type the word xterm? Because the convention exists to define the type of the component. It's a handy convention, and I'm betting there are a few people read…

So, if I don't know anything but Gnome terminal, and don't know what xterm is, if I see "xterm", I have to look it up. However, if I see "xterm(1)", I don't have to look it up?

Strange.

(And how did I get to the situation in which I know what (1) means, yet I only know Gnome terminal and don't know what xterm is?)

(What about the fact that xterm(1) is also a hyperlink in the sumitted page? You could change the anchor text to "xterm(foo)" and it would still navigate to the correct man page with one click.)

Re: Xterm(1) now UTF-8 by default on OpenBSD

#109

Post by zeveb » Wed, Mar 09, 2016, 3:23 PM UTC

alkonaut wrote: post #80

pilif wrote: post #39

Earlier quoted context omitted.

>Rob probably sees things like LANG and LC_ALL as bugs. His fix was UTF-8 everywhere, always The problems solved by LANG or LC_ALL are not solved by UTF8 alone. Even if you use UTF8 for all your input and output, there is still the question of how to format numbers and dates to the user and how to collate strings. These things are dependent on country and language, sometimes even varying between different places in a…

I still don't understand how encodings find their way into the localization. I understand that date/time/number formatting is localizable. Do not understand why "LC_TIME=en_GB.UTF-8" would be a different option from just "en_GB"? Can I really expect it to work if I set "LC_TIME=en_GB.encA" and "LC_MONETARY=en_GB.encB" How would the two encodings be used? How would they be used in a message consisting of both monetary…

What I hate is that the locales assume that date & number preferences are specific to one's physical location. I live in America, but I prefer German (9. March 2016 or 09.03.16) or British (9 March 2016 or 9/3/16) dates.

It's even worse when things assume that my date preferences reflect my unit preferences. I prefer standard units (feet, pounds, knots &c.) and British/Continental dates: I don't want to use French units, nor do I want to use American dates. And yet so much software assumes that it's all or nothing.

Re: Xterm(1) now UTF-8 by default on OpenBSD

#110

Post by Tiksi » Wed, Mar 09, 2016, 3:45 PM UTC

plugnburn wrote: post #77

Tiksi wrote: post #73

Earlier quoted context omitted.

And in 20-30 years we'll likely be saying the same about UTF-8. I figured that "ANSI" would give away that I wasn't being serious since it's not actually an encoding.

> And in 20-30 years we'll likely be saying the same about UTF-8. Well... If we will, why not? But the thing is that in 20-30 years we won't be able to invent any new writing systems that UTF-8 won't cover. Single-byte encodings were doomed because of their single-byteness. The same awaits two-byte encodings like UCS-2 (aka UTF-16BE) - we already have extended code points for something that glamour hipsters call "emo…

> But the thing is that in 20-30 years we won't be able to invent any new writing systems that UTF-8 won't cover.

I think you underestimate humanity's aptitude at creating things that don't fit into well defined standards.

My (admittedly poorly stated) point wasn't that we shouldn't be moving everything over to UTF-8. I personally use it wherever possible just because it makes life easier. My point was that there are decades of things that use ASCII-US or another one of the overlapping but incompatible encodings because they were the RightThing™ to use at the time and there's no way we're going to get rid of everything non-UTF-8 any time soon.

In 20-30 years we'll be saying "Why isn't everything in FutureText-64, it should be the only encoding. Why does anything else even exist?", and it'll be because we're saying the same about UTF-8 now.