Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

mattermost.com

Post reply on HN
Print view View original

11–20 of 115 posts

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#11

Post by tptacek »

ben509 wrote: post #5
jarym wrote: post #3

Glad this got found. I remember when XML was being widely adopted that there'd be frequent vulnerabilities found in Java-based parsers. A large part of this stems from how complicated XML can get - if it were only elements and attributes it might have been fine. Namespaces made it a bit more complicated. Processing Instructions made it hideous.

It's really nuts trying to implement something like SAML in XML. The standard is a security minefield.

What makes it worse is that XMLDSIG is exponentially more complicated. Most of the ecosystem literally shells out to libxmlsec1 and assumes it does the right thing. DSIG is a batshit standard that attempts to support arbitrary combinations of signed and unsigned parts in a single document, tied together with a DOM-like scheme, passed through a canonicalizing transformation that has itself broken SAML before. It's a fractal of bad security design.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#12

Post by Thaxll »

warp wrote: post #4

> Despite significant efforts by the Go security team, it has not been possible to patch the vulnerabilities discussed in this blog post. Well, that is not something you want to see in a public disclosure.

Because the standard lib was not designed to handle that use case and so they can't change it right now to not break compatibility, which is why they're going to add new API in Go 1.16 that will release in February.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#13

Post by Groxx »

Thaxll wrote: post #12
warp wrote: post #4

> Despite significant efforts by the Go security team, it has not been possible to patch the vulnerabilities discussed in this blog post. Well, that is not something you want to see in a public disclosure.

Because the standard lib was not designed to handle that use case and so they can't change it right now to not break compatibility, which is why they're going to add new API in Go 1.16 that will release in February.

Which they also point out may be insufficient:

>By Mattermost’s estimates this new API will not be a reasonable solution for most use cases currently affected by the vulnerabilities. Parsing and resolving namespaces is an essential requirement for correctly implementing SAML, and even considering only a limited set of real-world SAML messages without strict namespacing requirements would be unlikely to allow for a secure implementation.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#14

Post by tptacek »

politician wrote: post #8
jerf wrote: post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

Do you think there is a path forward for the Go team to release an XML library without namespace support that simply errors when they are encountered ("XML namespaces are considered harmful")?

You can do that, but SAML is heavily namespaced.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#15

Post by jerf »

politician wrote: post #8
jerf wrote: post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

Do you think there is a path forward for the Go team to release an XML library without namespace support that simply errors when they are encountered ("XML namespaces are considered harmful")?

They release something not called "encoding/xml". They could do what they did to the syscall package. The syscall package, by its nature, can't conform to the 1.0 compatibility promise Go itself maintains, because it changes outside of the scope of the Go project. So they froze the syscall package at some point, and then offered one in the golang.org/x/ namespace at https://pkg.go.dev/golang.org/x/sys .

I would again emphasize that encoding/xml, to my knowledge, only has problems with this particular roundtripping use case. It can consume non-namespaced XML correctly, and handle namespaced XML as long as you don't plan on re-emitting XML.

What would probably end up happening is a new package appearing on github.com for this use case, forked off of encoding/xml, for this use case. (If you're looking for a project that might attain some use, this is a likely candidate.) Unlike something like Python where the core packages are often C-based and thus you can expect better performance from the built-in "set" than somebody's pure-Python "set" implementation from before the built-in, encoding/xml is just a pile of pure Go code whose only advantage is that it ships with the compiler. Anyone can replace it without incurring any other disadvantage whenever they like.

(I looked a few versions ago, FWIW; encoding/xml has deviated so much from what I forked that my fork is essentially dead and no longer releasable without basically starting over from scratch. Plus I built it with the idea that it should be a minimal modification (so I could port it forward, which turned out to not work, but it's still how it was built)... if I was truly forking I'd have done some more extensive changes to it to support namespaces in general, rather than for my particular case.)

Anyhow, upshot, the Go project as a whole is not stuck... it is specifically encoding/xml as the standard, built-in library that is stuck. It's not like Go is completely incapable of handling XML correctly from first principles for some reason or anything.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#17

Post by tptacek »

Groxx wrote: post #13
Thaxll wrote: post #12

Earlier quoted context omitted.

Because the standard lib was not designed to handle that use case and so they can't change it right now to not break compatibility, which is why they're going to add new API in Go 1.16 that will release in February.

Which they also point out may be insufficient: > By Mattermost’s estimates this new API will not be a reasonable solution for most use cases currently affected by the vulnerabilities. Parsing and resolving namespaces is an essential requirement for correctly implementing SAML, and even considering only a limited set of real-world SAML messages without strict namespacing requirements would be unlikely to allow for a s…

Yes: people shouldn't be using encoding/xml to implement SAML, at all. The library was already functionally problematic for SAML, because it doesn't fully implement namespaces. Nor does it implement `xml-exc-c14n`. For the IdP I wrote last year, I just wrote my own XML; it's not that big a deal.

Software security people have understood for a long time that XMLDSIG is sketchy, and that implementations often need to be "bug-compatible" to interoperate safely. SAML is an XMLDSIG protocol. I feel bad for putting it this way, but I think that reasonably skilled security engineers should be alarmed if their platform's standard XML library easily allows you to implement something that claims to be DSIG.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#18

Post by dasyatidprime »

jerf wrote: post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

In other words, the minds of a large number of human specifiers and implementors have their own roundtrip corruption flaws where when you pass the XML namespaces specification through them, you get something out that's incoherent and doesn't interoperate properly, creating representation mismatch problems down the line back in the digital world.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#19

Post by jka »

tptacek wrote: post #17
Groxx wrote: post #13

Earlier quoted context omitted.

Which they also point out may be insufficient: > By Mattermost’s estimates this new API will not be a reasonable solution for most use cases currently affected by the vulnerabilities. Parsing and resolving namespaces is an essential requirement for correctly implementing SAML, and even considering only a limited set of real-world SAML messages without strict namespacing requirements would be unlikely to allow for a s…

Yes: people shouldn't be using encoding/xml to implement SAML, at all. The library was already functionally problematic for SAML, because it doesn't fully implement namespaces. Nor does it implement `xml-exc-c14n`. For the IdP I wrote last year, I just wrote my own XML; it's not that big a deal. Software security people have understood for a long time that XMLDSIG is sketchy, and that implementations often need to be…

Meta-question: are software standards generally becoming more security-friendly over time?

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#20

Post by tptacek »

jka wrote: post #19
tptacek wrote: post #17

Earlier quoted context omitted.

Yes: people shouldn't be using encoding/xml to implement SAML, at all. The library was already functionally problematic for SAML, because it doesn't fully implement namespaces. Nor does it implement `xml-exc-c14n`. For the IdP I wrote last year, I just wrote my own XML; it's not that big a deal. Software security people have understood for a long time that XMLDSIG is sketchy, and that implementations often need to be…

Meta-question: are software standards generally becoming more security-friendly over time?

No, but software itself is, because of designs like WireGuard (which aren't formal standards) and software like libsodium and signify.
Post reply on HN