Live data from Hacker News

Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

mattermost.com

101–110 of 115 posts

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#101
post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

You can see test cases: https://github.com/mattermost/xml-roundtrip-validator/blob/m...

Though they mention something called xml directive. I don't think such a thing exists.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#102
post #97

I've never liked (nor understood the popularity) of signature schemes that require parsing before verification. This has also led to problems with X.509. And DKIM. And plists. And package managers. And more. It's much simpler to sign the entire message, unparsed, and it's immune to these issues. We went through a decade of debate before deciding that "encrypt then mac" is the only right way to do things. That knowled…

as long as binary safe transport is given, possible. unless that ... . you can't skip the memory protocol.

signify works with text, you only decode the signature from base64 string and check the rest in raw.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#103
post #77

Earlier quoted context omitted.

Blog author here; Russell's implementation is backed by github.com/beevik/etree, but like you said, it's just an interface. The tokenizer is still encoding/xml. Adding better support for namespaces and providing APIs compatible with dsig doesn't remove the underlying vulnerabilities.

Huh, namespaces are a semantic convention on top of xml syntax, tokenizer can't really implement it.

Well that didn't stop them from trying

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#104
post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

Do you think there is a path forward for the Go team to release an XML library without namespace support that simply errors when they are encountered ("XML namespaces are considered harmful")?

No need to be something published by the Go team.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#105
post #77

Earlier quoted context omitted.

Blog author here; Russell's implementation is backed by github.com/beevik/etree, but like you said, it's just an interface. The tokenizer is still encoding/xml. Adding better support for namespaces and providing APIs compatible with dsig doesn't remove the underlying vulnerabilities.

Huh, namespaces are a semantic convention on top of xml syntax, tokenizer can't really implement it.

This is like saying that variable name scoping is a semantic convention on top of the C language grammar and that a lexer can't really implement it. In the case of C, it turns out that the lexer must implement it. In the case of XML, processing name spaces directives during lexing is the right thing to do in nearly all cases. But it's not what these SAML libraries needed.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#106
post #97

Earlier quoted context omitted.

as long as binary safe transport is given, possible. unless that ... . you can't skip the memory protocol.

signify works with text, you only decode the signature from base64 string and check the rest in raw.

yes, as long as the text is binary safe ... . XML specifies the character set for the document. IIRC also for base64.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#107
post #60

Earlier quoted context omitted.

Depending on who your customers are, you might get away with only supporting OIDC. But not supporting SAML is going to be a problem as you move into the big enterprises. Many big companies run on SAML, and expect to auth with vendors over SAML. That's why russell_h's comment is probably futile; it's the enterprises with the big SaaS budgets that keep SAML relevant, and they don't care if HN doesn't like it. Maybe in…

What about SCIM, "System for Cross-domain Identity Management", instead of SAML, if creating software for enterprises? SCIM: https://docs.microsoft.com/en-us/azure/active-directory/app-...

SCIM is a pleasure to implement compared to SAML, no doubt. You might be able to get away with only supporting SCIM, the main thing you'd be missing is "just-in-time" user provisioning.

But given that you'll probably need SCIM at some point anyway, probably a good idea to start with SCIM, and then add SAML only when you need to! It'll also inform what subset of SAML you actually need to implement.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#108
post #26
post #24

Earlier quoted context omitted.

> This breaks people's mental models of how XML works, and correspondingly, breaks people's code that manipulates XML. Because they usually have incorrect mental model. Blaming namespaces for name ambiguity would be the same as blaming the code "x = a + b" because "a" and "b" could be defined differently. Namespace prefixes are absolutely irrelevant, they only exists for your convenience.

That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not; instead, they are template variables, which can be shuffled throughout an XML document, requiring security software to constantly and reliably keep track of the value of the variable at multiple points. People sign URLs and JSON documents all…

In hindsight, it probably would have been better to define standard prefixes, let people just sort of register their own for non-standard ones in whatever manner is suitable for their particular top-level document type, and if somebody, somewhere out there did finally manage to stomp on each other, let that particular type of document where that happened deal with it.

While technically suboptimal compared to what currently exists, it would match people's expectations better, and in practice, I can't speak for everybody, but I just don't see a whole lot of documents with hundreds+ namespaces such that collisions are a realistic possibility. And when I do see documents with a lot of namespaces (XMPP, for instance, or XHTML+SVG+some other thing), there's still a top-level type that could keep its own registry just fine. A bit of guidance on naming extensions probably ("don't call it e:, work your name in somehow like with the initial of your company or something") would have 99.9% solved the problem.

Prior to seeing what happened I'd probably still have argued for the current namespaces spec. In principle it doesn't seem that complicated to me. But I'm obviously wrong in practice, because, like I said, I can hardly cite an example of them being used correctly at all.

(Likewise, in hindsight, entities shouldn't have been able to be recursive, and if we were spec'ing out the next generation of XML I'd straght-up remove them except for the ones necessary to XML itself, <, >, and & because UTF covers the major use case of entities now. I'd discard the "terrible, terrible templating language" use case entirely.)

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#109
post #108
post #26

Earlier quoted context omitted.

That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not; instead, they are template variables, which can be shuffled throughout an XML document, requiring security software to constantly and reliably keep track of the value of the variable at multiple points. People sign URLs and JSON documents all…

In hindsight, it probably would have been better to define standard prefixes, let people just sort of register their own for non-standard ones in whatever manner is suitable for their particular top-level document type, and if somebody, somewhere out there did finally manage to stomp on each other, let that particular type of document where that happened deal with it. While technically suboptimal compared to what cur…

In principle it doesn't seem that complicated to me. But I'm obviously wrong in practice, because, like I said, I can hardly cite an example of them being used correctly at all.

A snarky-but-mostly-true oversimplification: the complexity was necessary because XML was supposed to become a machine-readable interchange format for everything but it ended up not becoming that due to the complexity.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#110
post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

It is not in general legal to change prefixes and reserialize an XML document. Some official XML formats including XML Schema allow attribute values to reference prefixes in xs:QName types. One needs to bind the schema to detect that.

But it gets worth with XSLT using the prefixes in XPath expressions in attributes. If the prefixes are changed those values also need to be updated to change the prefix too, which requires complete knowledge of the format. This is because one cannot programmatically detect something like attributes that use custom data types that reference the prefixes in scope, but XSLT's xpath expressions show that W3C considers it legal to create such custom formats.

Post reply on HN