Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Glassworm is back: A new wave of invisible Unicode attacks hits repositories

aikido.dev

Post reply on HN
Print view View original

131–140 of 201 posts

Re: Glassworm is back: A new wave of invisible Unicode attacks hits repositories

#131

Post by slim »

WalterBright wrote: post #105
abujazar wrote: post #67

Earlier quoted context omitted.

You're talking about a subset of ASCII then. Unicode is supposed to support different languages and advanced typography, for which those characters are necessary. You can't write e.g. Arabic or Hebrew without those "unnecessary" invisible characters.

Please explain why an invisible zero width "character" is necessary.

if you write كلب which is an arabic word written right to left in the middle of an english sentence, you want to preserve the order of the characters in the stream for computer processing purposes. meaning the chararacter ك must come before the ل and after the e and the space with respect to the memory layout. whereas when displayed, it must be inverted to be legible. the solution is to have an invisible character that indicates a switch in text direction. if you were wondering, the situation where you want to write text in a foreign language within your text is very common outside english speaking countries.

Re: Glassworm is back: A new wave of invisible Unicode attacks hits repositories

#132

Post by ted_dunning »

WalterBright wrote: post #110
Yokohiii wrote: post #66

Earlier quoted context omitted.

What about numbers? Would they be assigned to arabic only? I guess someone will be offended by that. While at it we could also unify I, | and l. It's too confusing sometimes.

> While at it we could also unify I, | and l. It's too confusing sometimes. They render differently, so it's not a problem.

They only render differently in some fonts, on some displays.

Re: Glassworm is back: A new wave of invisible Unicode attacks hits repositories

#134

Post by ted_dunning »

WalterBright wrote: post #105
abujazar wrote: post #67

Earlier quoted context omitted.

You're talking about a subset of ASCII then. Unicode is supposed to support different languages and advanced typography, for which those characters are necessary. You can't write e.g. Arabic or Hebrew without those "unnecessary" invisible characters.

Please explain why an invisible zero width "character" is necessary.

To prevent ligatures from forming when you need that.

Re: Glassworm is back: A new wave of invisible Unicode attacks hits repositories

#135

Post by ted_dunning »

NoMoreNicksLeft wrote: post #76

Why can't code editors have a default-on feature where they show any invisible character (other than newlines)? I seem to remember Sublime doing this at least in some cases... the characters were rendered as a lozenge shape with the hex value of the character. Is there ever a circumstance where the invisible characters are both legitimate and you as a software developer wouldn't want to see them in the source code?

Check out emacs for options like this.

And, yes, there is a circumstance if you want to include Arabic or Hebrew in comments or strings. You need the zero width left-right markers to make that work.

Re: Glassworm is back: A new wave of invisible Unicode attacks hits repositories

#136

Post by andrewflnr »

godelski wrote: post #129
andrewflnr wrote: post #125

Earlier quoted context omitted.

They might say that your job is to make the product "better", and they might even think they mean it, but I think in practice you'll find that their definition of "better" as it relates to products is pretty closely related to money, and further that they are the authorities on what makes the product "better" so you should shut up and do what they say. If you want to make the product actually better, you're going to…

To be frank, I tried to address your point with my comment about the audience. I very much disagree that you start with money and work backwards to technical problems. I do not think this approach would make you efficient at solving problems nor at increasing profits for the business. And I still firmly believe they need us more than we need them. At the end of the day this is why they want AI coding agents to work o…

Talking about the audience completely misses my point. I'm not saying it's good to start with money and work back. I'm saying that's what companies actually do, and furthermore that's something the "dev audience" should understand about their employers.

> I do not think this approach would make you efficient at solving problems nor at increasing profits for the business.

If optimizing for profit doesn't result in profit, it's not the fault of the goal. That company was just incompetent. However many companies are, in fact, moderately competent, and optimizing for profit works fine for them. It even has a pretty heavy overlap with optimizing for good products, so that's nice.

It's fine. We agree on the ideal outcome in this situation.

Re: Glassworm is back: A new wave of invisible Unicode attacks hits repositories

#137

Post by krior »

WalterBright wrote: post #105
abujazar wrote: post #67

Earlier quoted context omitted.

You're talking about a subset of ASCII then. Unicode is supposed to support different languages and advanced typography, for which those characters are necessary. You can't write e.g. Arabic or Hebrew without those "unnecessary" invisible characters.

Please explain why an invisible zero width "character" is necessary.

To mark linewrapping-breakpoints in strings.

Re: Glassworm is back: A new wave of invisible Unicode attacks hits repositories

#139

Post by WalterBright »

jmusall wrote: post #117
WalterBright wrote: post #111

Earlier quoted context omitted.

Sometimes you gotta say no. Trying to please every hare brained idea leads to madness. Normalized code point sequences are another WTF feature.

Of course! I bet there are tons of ideas that didn't make it into Unicode, for better of worse. Where you draw the line is kind of arbitrary. You, personally, can of course opt out of all of that by restricting yourself to ASCII only, for example. But the rest of the world will continue to use Unicode.

> restricting yourself to ASCII only

My early compilers used code pages to work with Japanese, French and German customers. The original idea of Unicode was absolutely brilliant and I was all for it. D was an early total adopter of Unicode (C and C++ followed years later). I rejected code page support for D.

It's mission was to support all the letters in all the languages, which was a good straightforward mission. But then came fonts, formatting, layout, rendering, casing, sort ordering, normalization, combining, vote-for-my-letter-and-Ill-vote-for-yours, emoji, icons, semantic meanings, elvish, people who invent things and campaign to put them in so they'll leave a mark in history, ...

Re: Glassworm is back: A new wave of invisible Unicode attacks hits repositories

#140

Post by WalterBright »

ted_dunning wrote: post #128
WalterBright wrote: post #27

Earlier quoted context omitted.

Another dum dum Unicode idea is having multiple code points with identical glyphs. Rule of thumb: two Unicode sequences that look identical when printed should consist of the same code points.

One of the ground rules of Unicode is the round trip rule. You have to be able to translate to and from Unicode without loss of information.

They threw that out the window with normalization.
Post reply on HN