Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why did the OpenSSL punycode vulnerability happen?

words.filippo.io

Post reply on HN
Print view View original

21–30 of 104 posts

Re: Why did the OpenSSL punycode vulnerability happen?

#21

Post by tester756 »

0xbadcafebee wrote: post #12
tester756 wrote: post #9

Earlier quoted context omitted.

But why? where does the complexity come from

The first level of complexity comes from the format. A bit array is super easy to parse (in C, and assuming you take care of endianness). JSON is more complicated; YAML is more complicated than JSON; XML is more complicated than YAML; X.509 is more complicated than XML (I think, anyway). The more complex the data format, the more complex the parsing; the more complex; the more opportunity for bugs. The second level o…

>"Why does it seem easier in high-level languages?" High-level languages have slowly had their bugs stripped out, and give you features that are rarer in low-level languages. You literally aren't writing the same routines in high-level languages because you don't need to. If you had to do all the same things, you'd have the same bugs. And a lot of newbies simply are lucky and don't personally run into the bugs that are already there.

Which bugs precisely are you talking about?

Sane string implementation, so instead of performing some shenanigans with buffers to concat two strings, I can just "a" + "b"?

>If you had to do all the same things, you'd have the same bugs.

Why in lower level languages people cannot write some handy abstractions which will result in better security and dev. experience?

Re: Why did the OpenSSL punycode vulnerability happen?

#22

Post by alserio »

tester756 wrote: post #19
alserio wrote: post #18

Earlier quoted context omitted.

I'll give you a tautological and useless answer. I hope you don't mind. A parser is a kind of interpreter, where code is executed based on external input. When user input controls how code is executed you have opened the doors of hell: it's hard to guarantee that all of the possible executions are safe.

But why? Parsers executes safe code fragments in order based on the input How safe operations result in unsafe results? I'm not talking about side channels here.

Well, I see two ways it might be: the operations are not really safe in an absolute sense or the operations composition does not preserve safety. Or you could say that an operation can be safe in a context but ops+context does not compose as well as we would like to guarantee the preservation of safety

Re: Why did the OpenSSL punycode vulnerability happen?

#23

Post by insanitybit »

tester756 wrote: post #17
insanitybit wrote: post #13

Earlier quoted context omitted.

I think it's easy to hand write a parser that can get into a weird state. Some formats are also easier to write a parser for safely vs others. For example, if you have a Tag Length Value data format that says "5ABCDE" maybe that means "the next 5 bytes are a string, then there's something else afterwards or it's the end", you don't want to allocate a buffer of 5 values and just keep writing into it without checking t…

>Doing this sort of thing efficiently may also mean that you're tempted to remove something like a bounds check. After all, if you already know that the value is 5 bytes, why check on every access? People really want parsers to be fast. In what kind of software bounds check have this significant perf. penalty? The only people I've heard talking about such a stuff were firmware devs. >Further, in C, buffers don't have…

> In what kind of software bounds check have this significant perf. penalty?

All kinds. But a lot of it is just that parsers are extremely easy to benchmark and benchmarks promote optimization.

> Cannot C have some wrapper over those poor strings

Sure, but there's nothing native.

Re: Why did the OpenSSL punycode vulnerability happen?

#24

Post by green_on_black »

tester756 wrote: post #7

It feels like issues like those are more common in parsers, this specific kind of software. But why? Why is parsing so hard? or is it just in low lvl languages? or maybe languages with poor string primitives? I've written parsers in high level languages and it didnt felt dangerous or insanely hard

tl;dr: Using C/C++ and being human => memory safety problems.

Re: Why did the OpenSSL punycode vulnerability happen?

#25

Post by mikepavone »

> There is a function, ossl_punycode_decode that decodes Punycode. Punycode is a way to encode Unicode as ASCII, used to represent Unicode strings in the ASCII-only world of DNS. ossl_punycode_decode takes an output buffer, and if the buffer runs out it keeps parsing and verifying the Punycode but discards the rest of the output.

> I did not have the heart to figure out why it works like this. Maybe there's a good reason to do progressive parsing. Maybe it's an artifact of how C makes you do memory management or of OpenSSL's C style. Anyway.

Not involved in OpenSSL, but this is a fairly common pattern in a lot of C APIs. You want to decode some data, but you're not sure how big the output is going to be ahead of time. You could write a separate function to calculate the length, but that function has to do most of the work of actual decoding to figure out that length. A lot of times the output is small enough it can fit in some conservatively sized buffer so you can save a fair bit of work by having a (potentially stack-allocated) buffer of some fixed size and then allocating a precisely sized buffer on the heap if it turns out to not be big enough. Further, having a separate length function means you typically end up with two similar but separate decode implementations which has its own problems.

In most other languages, you have some sort of growable container in the standard library so you just avoid the problem entirely at the expense of having less control over memory allocations.

Re: Why did the OpenSSL punycode vulnerability happen?

#26

Post by tester756 »

insanitybit wrote: post #23
tester756 wrote: post #17

Earlier quoted context omitted.

>Doing this sort of thing efficiently may also mean that you're tempted to remove something like a bounds check. After all, if you already know that the value is 5 bytes, why check on every access? People really want parsers to be fast. In what kind of software bounds check have this significant perf. penalty? The only people I've heard talking about such a stuff were firmware devs. >Further, in C, buffers don't have…

> In what kind of software bounds check have this significant perf. penalty? All kinds. But a lot of it is just that parsers are extremely easy to benchmark and benchmarks promote optimization. > Cannot C have some wrapper over those poor strings Sure, but there's nothing native.

>Sure, but there's nothing native.

But why? strings are used by all programmers everyday

I struggle to understand why you wouldn't want to make them state of the art, or decent at least.

Re: Why did the OpenSSL punycode vulnerability happen?

#27

Post by insanitybit »

tester756 wrote: post #26
insanitybit wrote: post #23

Earlier quoted context omitted.

> In what kind of software bounds check have this significant perf. penalty? All kinds. But a lot of it is just that parsers are extremely easy to benchmark and benchmarks promote optimization. > Cannot C have some wrapper over those poor strings Sure, but there's nothing native.

>Sure, but there's nothing native. But why? strings are used by all programmers everyday I struggle to understand why you wouldn't want to make them state of the art, or decent at least.

C is an extremely conservative language. You don't even get booleans unless you're in C99 and import it.

Re: Why did the OpenSSL punycode vulnerability happen?

#28

Post by AceJohnny2 »

> As curl author Daniel Stenberg said, "I've never even considered to decode punycode. Why does something like OpenSSL need to decode this?"

> [...]

> Internationalization is not the issue, internationalization is the job.

I want to cheer at this.

Internationalization is hard, really hard, especially in a computing world so defined by its english-language dominance. The little amount of effort demonstrated in defining and testing this feature demonstrates that.

For all I admire Stenberg's work and focus on quality, that was a rather poor take from him.

Re: Why did the OpenSSL punycode vulnerability happen?

#29

Post by j-bos »

mikepavone wrote: post #25

> There is a function, ossl_punycode_decode that decodes Punycode. Punycode is a way to encode Unicode as ASCII, used to represent Unicode strings in the ASCII-only world of DNS. ossl_punycode_decode takes an output buffer, and if the buffer runs out it keeps parsing and verifying the Punycode but discards the rest of the output. > I did not have the heart to figure out why it works like this. Maybe there's a good re…

I am by no means a rust developer and asking in ignorance, would this have been a problem had it been written in rust?

Re: Why did the OpenSSL punycode vulnerability happen?

#30

Post by thephyber »

tester756 wrote: post #26
insanitybit wrote: post #23

Earlier quoted context omitted.

> In what kind of software bounds check have this significant perf. penalty? All kinds. But a lot of it is just that parsers are extremely easy to benchmark and benchmarks promote optimization. > Cannot C have some wrapper over those poor strings Sure, but there's nothing native.

>Sure, but there's nothing native. But why? strings are used by all programmers everyday I struggle to understand why you wouldn't want to make them state of the art, or decent at least.

> > Sure, but there's nothing native.

> But why? strings are used by all programmers everyday

Different kinds of strings can have extremely different performance profiles.

A statically allocated string of ASCII (single byte) characters.

A dynamically allocated string of unicode (multibyte) characters where the length of allocation is not known ahead of time is very different. C requires the developer to know the differences and knows how to deal with them treat them.

Post reply on HN