Why did the OpenSSL punycode vulnerability happen? • Hacker News Board

Re: Why did the OpenSSL punycode vulnerability happen?

#31

Post by toast0 » Thu, Nov 03, 2022, 12:21 AM UTC

alserio wrote: post #18

tester756 wrote: post #7

It feels like issues like those are more common in parsers, this specific kind of software. But why? Why is parsing so hard? or is it just in low lvl languages? or maybe languages with poor string primitives? I've written parsers in high level languages and it didnt felt dangerous or insanely hard

I'll give you a tautological and useless answer. I hope you don't mind. A parser is a kind of interpreter, where code is executed based on external input. When user input controls how code is executed you have opened the doors of hell: it's hard to guarantee that all of the possible executions are safe.

Here's an even more tautological answer. You can't misparse input unless you're parsing input. That's why parsers have trouble with parsing input.

Re: Why did the OpenSSL punycode vulnerability happen?

#32

Post by FiloSottile » Thu, Nov 03, 2022, 12:27 AM UTC

AceJohnny2 wrote: post #28

> As curl author Daniel Stenberg said, "I've never even considered to decode punycode. Why does something like OpenSSL need to decode this?" > [...] > Internationalization is not the issue, internationalization is the job. I want to cheer at this. Internationalization is hard , really hard, especially in a computing world so defined by its english-language dominance. The little amount of effort demonstrated in defini…

Oh I don't think Daniel was asking why we're doing i18n. My remark was not at him, at all. I was just pre-empting a possible made up objection. He's in fact correct to wonder why punycode decoding ended up in OpenSSL, as the rest of that section explores. The point being that OpenSSL could do its job and still support i18n domains and emails without having to ever decode punycode if only the spec had made different tradeoffs.

Re: Why did the OpenSSL punycode vulnerability happen?

#33

Post by vore » Thu, Nov 03, 2022, 12:31 AM UTC

j-bos wrote: post #29

mikepavone wrote: post #25

> There is a function, ossl_punycode_decode that decodes Punycode. Punycode is a way to encode Unicode as ASCII, used to represent Unicode strings in the ASCII-only world of DNS. ossl_punycode_decode takes an output buffer, and if the buffer runs out it keeps parsing and verifying the Punycode but discards the rest of the output. > I did not have the heart to figure out why it works like this. Maybe there's a good re…

I am by no means a rust developer and asking in ignorance, would this have been a problem had it been written in rust?

Rust would take the growable container approach here, so it wouldn't be vulnerable in the same way. There's nothing in C that prevents you from doing this either, but as the parent post mentions it's common style to do this (see also things like snprintf).

Re: Why did the OpenSSL punycode vulnerability happen?

#34

Post by mike_d » Thu, Nov 03, 2022, 12:32 AM UTC

j-bos wrote: post #29

mikepavone wrote: post #25

> There is a function, ossl_punycode_decode that decodes Punycode. Punycode is a way to encode Unicode as ASCII, used to represent Unicode strings in the ASCII-only world of DNS. ossl_punycode_decode takes an output buffer, and if the buffer runs out it keeps parsing and verifying the Punycode but discards the rest of the output. > I did not have the heart to figure out why it works like this. Maybe there's a good re…

I am by no means a rust developer and asking in ignorance, would this have been a problem had it been written in rust?

It is an exercise in navel gazing until an organization steps up to fund: 1) a linkable clib harness that exports Rust functions to replace OpenSSL functions. and 2) pays to get the resulting harness and underlying code FIPS certified so people can use it.

Re: Why did the OpenSSL punycode vulnerability happen?

#35

Post by Ygg2 » Thu, Nov 03, 2022, 12:39 AM UTC

j-bos wrote: post #29

mikepavone wrote: post #25

> There is a function, ossl_punycode_decode that decodes Punycode. Punycode is a way to encode Unicode as ASCII, used to represent Unicode strings in the ASCII-only world of DNS. ossl_punycode_decode takes an output buffer, and if the buffer runs out it keeps parsing and verifying the Punycode but discards the rest of the output. > I did not have the heart to figure out why it works like this. Maybe there's a good re…

I am by no means a rust developer and asking in ignorance, would this have been a problem had it been written in rust?

Not in this case as far as I can tell.

Re: Why did the OpenSSL punycode vulnerability happen?

#36

Post by pshc » Thu, Nov 03, 2022, 1:07 AM UTC

tester756 wrote: post #7

It feels like issues like those are more common in parsers, this specific kind of software. But why? Why is parsing so hard? or is it just in low lvl languages? or maybe languages with poor string primitives? I've written parsers in high level languages and it didnt felt dangerous or insanely hard

Parsing isn’t all that hard, it’s about working with the right tools and level of abstractions. Messing around in C with null-terminated strings is just hilariously error prone.

Re: Why did the OpenSSL punycode vulnerability happen?

#37

Post by newpavlov » Thu, Nov 03, 2022, 1:38 AM UTC

j-bos wrote: post #29

mikepavone wrote: post #25

> There is a function, ossl_punycode_decode that decodes Punycode. Punycode is a way to encode Unicode as ASCII, used to represent Unicode strings in the ASCII-only world of DNS. ossl_punycode_decode takes an output buffer, and if the buffer runs out it keeps parsing and verifying the Punycode but discards the rest of the output. > I did not have the heart to figure out why it works like this. Maybe there's a good re…

I am by no means a rust developer and asking in ignorance, would this have been a problem had it been written in rust?

Rust code would probably return an error on output buffer overflow instead of siltently discarding data. Something like this: https://docs.rs/base16ct/latest/base16ct/mixed/fn.decode.htm...

Re: Why did the OpenSSL punycode vulnerability happen?

#38

Post by cratermoon » Thu, Nov 03, 2022, 1:47 AM UTC

green_on_black wrote: post #24

tester756 wrote: post #7

It feels like issues like those are more common in parsers, this specific kind of software. But why? Why is parsing so hard? or is it just in low lvl languages? or maybe languages with poor string primitives? I've written parsers in high level languages and it didnt felt dangerous or insanely hard

tl;dr: Using C/C++ and being human => memory safety problems.

Also, as we learned back when Heartbleed was discovered, the OpenSSL code is not in good shape. It "suffers from maintenance", as one clever wag said about legacy code. There's a reason LibreSSL forked the code. More distributions need to switch away from OpenSSL.

And before anyone pipes up, I'm not claiming LibreSSL does not and will not ever haver vulnerabilities. I'm saying that ripping stuff like punycode out of the library reduces the attack surface. https://isc.sans.edu/diary/rss/29208

Re: Why did the OpenSSL punycode vulnerability happen?

#39

Post by simcop2387 » Thu, Nov 03, 2022, 1:51 AM UTC

j-bos wrote: post #29

mikepavone wrote: post #25

> There is a function, ossl_punycode_decode that decodes Punycode. Punycode is a way to encode Unicode as ASCII, used to represent Unicode strings in the ASCII-only world of DNS. ossl_punycode_decode takes an output buffer, and if the buffer runs out it keeps parsing and verifying the Punycode but discards the rest of the output. > I did not have the heart to figure out why it works like this. Maybe there's a good re…

I am by no means a rust developer and asking in ignorance, would this have been a problem had it been written in rust?

Probably not. It's definitely possible to write this function exactly as it is in openssl with this issue, nothing about Rust's safety claims prevent it. However, Rust doesn't have the same issues that lead to this kind of pattern being common like they are in C. Result types being so common, better handling of dynamic allocations, and at least for now a generally (it can't last forever) more security conscious community mean that other patterns will be chosen unstead. I'm not sure any language can actually prevent it from being possible outright, but you possibly pair it with some kind of formal proof system like using Coq and TLA+ might let you prove thag an issue like this doesn't exist in a specific implementation. However it's incredibly hard to do this.

Re: Why did the OpenSSL punycode vulnerability happen?

#40

Post by Matheus28 » Thu, Nov 03, 2022, 2:02 AM UTC

tester756 wrote: post #21

0xbadcafebee wrote: post #12

Earlier quoted context omitted.

The first level of complexity comes from the format. A bit array is super easy to parse (in C, and assuming you take care of endianness). JSON is more complicated; YAML is more complicated than JSON; XML is more complicated than YAML; X.509 is more complicated than XML (I think, anyway). The more complex the data format, the more complex the parsing; the more complex; the more opportunity for bugs. The second level o…

>"Why does it seem easier in high-level languages?" High-level languages have slowly had their bugs stripped out, and give you features that are rarer in low-level languages. You literally aren't writing the same routines in high-level languages because you don't need to. If you had to do all the same things, you'd have the same bugs. And a lot of newbies simply are lucky and don't personally run into the bugs that a…

>Sane string implementation, so instead of performing some shenanigans with buffers to concat two strings, I can just "a" + "b"?

Because system language users care about what happens when you do that. Where is it being allocated? What happens to the original strings? And a lot of other questions that relate to memory management. These languages are faster in part because of the control you get over allocations.

In C++ one could use a vector to build a temporary string (or use stringstream for a fancier interface), but that also means that whenever something is appended to it, it must check the container capacity to handle possible reallocations, etc. Very similar to how most high level languages handle strings. But that comes at a cost.

A common C pattern is to simply receive a pointer to a block of memory to use for the string, write to that and null terminate it. If the buffer size is insufficient, it will stop writing but continue parsing so it can return the size of the required buffer so the user can allocate that and call the function again. Most libraries avoid allocating stuff on behalf of the user. On the common case (buffer size is enough), it'll be much faster as no heap memory needs be allocated, moved around, etc.