Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

mattermost.com

Post reply on HN
Print view View original

41–50 of 115 posts

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#41

Post by tptacek »

ivanbakel wrote: post #36
tptacek wrote: post #10

Earlier quoted context omitted.

It's a little-loved library in the standard library that wasn't designed to support cryptographic security, but was by many projects repurposed as such to support SAML. It was a mistake for any SAML project to depend on encoding/xml.

> that wasn't designed to support cryptographic security An XML library doesn't have to support cryptographic security - it just has to perform XML en/decoding effectively. How can it be a mistake for a project to rely on part of the standard library?

You write this as if XMLDSIG was straightforward, but people who have worked with XMLDSIG before know that it is not, and people who haven't worked with XMLDSIG should know that they need to research new cryptosystems before slapping them together out of spare parts.

Just to clear it up and state it plainly: it is never reasonable to assume that a given XML library is suitable for building XMLDSIG on top of or alongside.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#42

Post by tptacek »

xyzzy_plugh wrote: post #38
tptacek wrote: post #10

Earlier quoted context omitted.

It's a little-loved library in the standard library that wasn't designed to support cryptographic security, but was by many projects repurposed as such to support SAML. It was a mistake for any SAML project to depend on encoding/xml.

I don't doubt it was a mistake, but I'm curious what the alternative is here? I'm looking to add SAML to my to project in the medium-term.

SAML is the only mainstream user of XMLDSIG and 99%+ of the installed base of XMLDSIG. SAML libraries should include purpose-built, locked-down, SAML-only XMLDSIGs, and those XMLDSIGs should include purpose-built, stripped-down XMLs.

The XML isn't even the hard problem here! XMLDSIG and XML Canonicalization are much more complicated than the baseline XML parser.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#43

Post by jiggawatts »

ben509 wrote: post #5
jarym wrote: post #3

Glad this got found. I remember when XML was being widely adopted that there'd be frequent vulnerabilities found in Java-based parsers. A large part of this stems from how complicated XML can get - if it were only elements and attributes it might have been fine. Namespaces made it a bit more complicated. Processing Instructions made it hideous.

It's really nuts trying to implement something like SAML in XML. The standard is a security minefield.

I've been cavorting around that minefield recently. I still have some of my legs and a tiny bit of my sanity.

The most recent "fun" I had was that on a Citrix NetScaler, if you enable a certain n-Factor workflow, it sends a SAML request to the IdP that Microsoft products only reject as "invalid XML".

From what I can gather the XML being sent is perfectly valid. The issue must be something hideously subtle, like the white space or UTF-8 encoding being subtly different that is upsetting the Microsoft SAML implementations, but not any others.

Have a look at some SAML XML examples online: https://www.samltool.com/generic_sso_res.php

They're hideous not because they're XML, but because they're bad XML! The SAML standard defines its own "namespace attributes" separately but on top of the XML namespaces!

Similarly, instead of the straightforward way to encode the data:

    value
They abstract one level up unnecessarily:

    
        attr
        value
This is the same mistake people make in database schema design, where they'll have a table with columns called "Key", "ColumnName", and "ColumnValue".

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#44

Post by eznzt »

tptacek wrote: post #41
ivanbakel wrote: post #36

Earlier quoted context omitted.

> that wasn't designed to support cryptographic security An XML library doesn't have to support cryptographic security - it just has to perform XML en/decoding effectively. How can it be a mistake for a project to rely on part of the standard library?

You write this as if XMLDSIG was straightforward, but people who have worked with XMLDSIG before know that it is not, and people who haven't worked with XMLDSIG should know that they need to research new cryptosystems before slapping them together out of spare parts. Just to clear it up and state it plainly: it is never reasonable to assume that a given XML library is suitable for building XMLDSIG on top of or alongs…

I don't know what I'm talking about, but if the XMLDSIG support of encoding/xml does not work properly and no one can fix it, can't they just drop it?

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#45

Post by lyxsus »

tptacek wrote: post #40
lyxsus wrote: post #33

Earlier quoted context omitted.

> That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not; I'm not sure what you mean by that, tbh. It seems to me that namespace expansion is absolutely straightforward and deterministic. There're scopes, yes, but they're too well-defined (if that's what you mean).

Yes, you are describing the same feature I am with slightly different words. It obviously causes problems. You could describe XML entity expansion in simple terms too, and it would remain one of the major causes of game-over vulnerabilities in enterprise software over the last decade.

Well, yeah, true.

I believe it's mostly implementation and popularisation problems.

The w3c specs surrounding xml/xpath/xslt/rdf and etc are very well designed but it's possible to appreciate them only after you spend ridiculously unreasonable amount of time reading and putting them all together. Otherwise it looks like a stupid pile of complexity with no purpose.

And what upsets me the most is the lack of really good libraries, everything I worked with just sucks so much.

I still have a hope that maybe in 5-15 years things will change.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#46

Post by tptacek »

eznzt wrote: post #44
tptacek wrote: post #41

Earlier quoted context omitted.

You write this as if XMLDSIG was straightforward, but people who have worked with XMLDSIG before know that it is not, and people who haven't worked with XMLDSIG should know that they need to research new cryptosystems before slapping them together out of spare parts. Just to clear it up and state it plainly: it is never reasonable to assume that a given XML library is suitable for building XMLDSIG on top of or alongs…

I don't know what I'm talking about, but if the XMLDSIG support of encoding/xml does not work properly and no one can fix it, can't they just drop it?

Drop what? encoding/xml was never a reasonable building block for SAML and XMLDSIG, that was pretty immediately apparent from the library itself, and the Go project never told people they should using encoding/xml this way. They never picked it up in the first place, is what I'm saying.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#47

Post by hnarn »

dboreham wrote: post #39
tedunangst wrote: post #7

I've never liked (nor understood the popularity) of signature schemes that require parsing before verification. This has also led to problems with X.509. And DKIM. And plists. And package managers. And more. It's much simpler to sign the entire message, unparsed, and it's immune to these issues. We went through a decade of debate before deciding that "encrypt then mac" is the only right way to do things. That knowled…

I believe the problem being addressed is where the payload may be transcoded in flight or otherwise not delivered in exactly the same form. Put another way: the signature validates the payload, however it may end up being represented to the validator on delivery. It isn't simply a transport integrity measure.

I might be missing the point here, but isn't the whole idea of signing a message that it should not be possible to "transcode the message in flight"? If you even allow the message to be "not delivered in exactly the same form" in the first place, you're introducing an attack vector completely without reason, because what you instead could do is let the payload be strongly signed and unchanged, and then have differing parsing rules at the end.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#48

Post by neilv »

tptacek wrote: post #42
xyzzy_plugh wrote: post #38

Earlier quoted context omitted.

I don't doubt it was a mistake, but I'm curious what the alternative is here? I'm looking to add SAML to my to project in the medium-term.

SAML is the only mainstream user of XMLDSIG and 99%+ of the installed base of XMLDSIG. SAML libraries should include purpose-built, locked-down, SAML-only XMLDSIGs, and those XMLDSIGs should include purpose-built, stripped-down XMLs. The XML isn't even the hard problem here! XMLDSIG and XML Canonicalization are much more complicated than the baseline XML parser.

Yep, I've implemented SAML multiple times with a purpose-built processing.

And I implemented a non-SAML use of XML-DSIG standards, and discovered, when attempting to interoperate, that a major platform vendor's implementation of wasn't compliant, such that hashes would only be correct using that vendor's implementation (which I initially assumed was a mistake of mine, until an expert confirmed the major vendor was actually wrong).

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#49

Post by sleepydog »

politician wrote: post #8
jerf wrote: post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

Do you think there is a path forward for the Go team to release an XML library without namespace support that simply errors when they are encountered ("XML namespaces are considered harmful")?

XML namespaces are ubiquitous. The utility of such a library would be very questionable.

While they do have the problems described, XML namespaces are what allow for abstraction and composition of documents from disparate systems.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#50

Post by progval »

tptacek wrote: post #26
lyxsus wrote: post #24

Earlier quoted context omitted.

> This breaks people's mental models of how XML works, and correspondingly, breaks people's code that manipulates XML. Because they usually have incorrect mental model. Blaming namespaces for name ambiguity would be the same as blaming the code "x = a + b" because "a" and "b" could be defined differently. Namespace prefixes are absolutely irrelevant, they only exists for your convenience.

That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not; instead, they are template variables, which can be shuffled throughout an XML document, requiring security software to constantly and reliably keep track of the value of the variable at multiple points. People sign URLs and JSON documents all…

> instead, they are template variables, which can be shuffled throughout an XML document, requiring security software to constantly and reliably keep track of the value of the variable at multiple points

Isn't the issue here that they are mixing this templating with the business logic? They should be fine if the XML parser (or some post-processing) expanded the namespaces and business logic didn't see them at all.

> People sign URLs and JSON documents all the time with schemes that don't have this goofy property.

Similarly, that might be a design issue. They should only sign documents they 100% built and serialized themselves, so the set of tags and namespaces.

Post reply on HN