Live data from Hacker News

Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

mattermost.com

31–40 of 115 posts

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#31
post #25

Can we just consider XML itself an unsolvable vulnerability and call it day?

What do you suggest to use instead? It's a bit too early to use RDF, people haven't catch up yet.

There is no good reason that the thing that 98% of people use SAML to do should involve XML.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#32
XML namespaces were controversial when introduced, and their implementation as privileged "xmlns:..." attributes with complex scoping, layering, and defaulting rules have been criticized many times; see [1] for a reflection from 2010 by an insider admitting to the fact that "every step on the process that led to the current situation with XML Namespaces seems reasonable".

When in 1996-98 W3C/The SGML Extended Review Board subset XML from SGML to define a generic markup convention for use with the expected wealth of upcoming vocabularies on the web, the issue of name collisions between elements (and attributes) from different vocabularies was deemed significant. Of course, in hindsight, with only SVG and MathML (and rarely HTML 5 in XHTML serialization) left on the web and having been incorporated as foreign elements directly into HTML, this seems overkill (even though there are actually collisions between eg. the title element in SVG vs HTML).

There's an alternative (and saner IMHO) approach for dealing with XML namespaces in ISO/IEC 19757-9 [2] by just presenting a canonical (ie. always the same) namespace prefix as part of an element name by a parser API to an app, guided by processing instructions for binding canonical namespace prefixes to namespace URLs, which might also help enterprise-y XML with lots of XML Schema use. Of course, this doesn't help with roundtripping xmlns-bindings (eg. with their exact ordering, possible redundancy, temporary/insignificant namespace prefixes, re-binding in document fragments etc.) through DOM representations, which seems the problem here.

[1]: https://blog.jclark.com/2010/01/xml-namespaces.html

[2]: https://www.iso.org/obp/ui/#iso:std:iso-iec:19757:-9:ed-1:v1...

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#33
post #26
post #24

Earlier quoted context omitted.

> This breaks people's mental models of how XML works, and correspondingly, breaks people's code that manipulates XML. Because they usually have incorrect mental model. Blaming namespaces for name ambiguity would be the same as blaming the code "x = a + b" because "a" and "b" could be defined differently. Namespace prefixes are absolutely irrelevant, they only exists for your convenience.

That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not; instead, they are template variables, which can be shuffled throughout an XML document, requiring security software to constantly and reliably keep track of the value of the variable at multiple points. People sign URLs and JSON documents all…

> That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not;

I'm not sure what you mean by that, tbh. It seems to me that namespace expansion is absolutely straightforward and deterministic. There're scopes, yes, but they're too well-defined (if that's what you mean).

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#34
post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

In other words, the minds of a large number of human specifiers and implementors have their own roundtrip corruption flaws where when you pass the XML namespaces specification through them, you get something out that's incoherent and doesn't interoperate properly, creating representation mismatch problems down the line back in the digital world.

To add to this with some (potentially out-of-date!) personal experience: sometime around 2008, now somewhere in my dusty directories, I started implementing my own Ruby XML DOM-like library (based on the Parsifal XML lexer in C) mainly to handle this properly, because the most “normal” REXML did something really horrible in its API for namespaced attributes that made them almost unusable by clients of the library. (I forget why I couldn't use the Ruby bindings to libxml2 at the time; I think maybe they didn't support several things I wanted to do, and I couldn't add them without patching and vendoring libxml2, and that would be its own disaster because of shared libraries, etc.) Specifically, attribute accesses didn't always work consistently with namespaces, the callers had to handle prefix management themselves in places, and looping over the attributes of an element required you to handle both “single” attributes and sub-collections of namespaced attributes if there were any qualified and unqualified attributes with the same short name, because the second was what the outer collection was indexed by…

(I originally wrote this comment with a more fully worked-out example, but after viewing it in context I realized it was way too long to be an only-partly-on-topic comment on this thread, so I'll probably move it to a post elsewhere and submit it later.)

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#35
post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

> I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they could be. In an XML document using XML namespaces, "ns1:tagname" may not equal "ns1:tagnam…

Not round-tripping prefixes breaks other things, and this is around where I start more thoroughly agreeing that XML namespaces tried to do too much at once: mainly, there are XML-based definitions, I think including XPath and XSLT's use of it (or old XSLT's use of what might not have been XPath yet?), that use the ambient namespace prefix set as context for interpreting attribute values that contain XML names. If you might ever interchange some prefixes, you might even get similar “looks valid but means the wrong thing” problems.

And if you try to combine namespaces with DTDs (which is just an explosive mix to start with, and I think is just recommended to never do) you get other problems, because you're no longer allowed to add arbitrary namespace declarations in the middle, so anything that round-trips prefixes but might ever add redundant declarations of them won't reliably produce something that DTD-validates, and if you're transforming into a DTD from something that might have used other namespaces, you have to make sure to remove all the extra declarations, and…

Note that most of this is still “well-defined”, it's just awkwardly hairy. This is not to be taken as an excuse to implement the standard badly or incorrectly if you're going to handle it at all.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#36
post #10
post #4

> Despite significant efforts by the Go security team, it has not been possible to patch the vulnerabilities discussed in this blog post. Well, that is not something you want to see in a public disclosure.

It's a little-loved library in the standard library that wasn't designed to support cryptographic security, but was by many projects repurposed as such to support SAML. It was a mistake for any SAML project to depend on encoding/xml.

> that wasn't designed to support cryptographic security

An XML library doesn't have to support cryptographic security - it just has to perform XML en/decoding effectively. How can it be a mistake for a project to rely on part of the standard library?

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#37
`encoding/xml` has had broken handling of namespaces for a long time. It’s possible to hack it on but the only reasonable choice is to use a libxml2 binding which also gets you canonicalization, another can of worms.

Unsurprised it can cause security issues, especially in XML-DSig which is a nightmare to handle correctly.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#38
post #10
post #4

> Despite significant efforts by the Go security team, it has not been possible to patch the vulnerabilities discussed in this blog post. Well, that is not something you want to see in a public disclosure.

It's a little-loved library in the standard library that wasn't designed to support cryptographic security, but was by many projects repurposed as such to support SAML. It was a mistake for any SAML project to depend on encoding/xml.

I don't doubt it was a mistake, but I'm curious what the alternative is here? I'm looking to add SAML to my to project in the medium-term.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#39

I've never liked (nor understood the popularity) of signature schemes that require parsing before verification. This has also led to problems with X.509. And DKIM. And plists. And package managers. And more. It's much simpler to sign the entire message, unparsed, and it's immune to these issues. We went through a decade of debate before deciding that "encrypt then mac" is the only right way to do things. That knowled…

I believe the problem being addressed is where the payload may be transcoded in flight or otherwise not delivered in exactly the same form. Put another way: the signature validates the payload, however it may end up being represented to the validator on delivery. It isn't simply a transport integrity measure.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#40
post #33
post #26

Earlier quoted context omitted.

That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not; instead, they are template variables, which can be shuffled throughout an XML document, requiring security software to constantly and reliably keep track of the value of the variable at multiple points. People sign URLs and JSON documents all…

> That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not; I'm not sure what you mean by that, tbh. It seems to me that namespace expansion is absolutely straightforward and deterministic. There're scopes, yes, but they're too well-defined (if that's what you mean).

Yes, you are describing the same feature I am with slightly different words. It obviously causes problems. You could describe XML entity expansion in simple terms too, and it would remain one of the major causes of game-over vulnerabilities in enterprise software over the last decade.
Post reply on HN