Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

mattermost.com

Post reply on HN
Print view View original

21–30 of 115 posts

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#21

Post by tptacek »

jerf wrote: post #15
politician wrote: post #8

Earlier quoted context omitted.

Do you think there is a path forward for the Go team to release an XML library without namespace support that simply errors when they are encountered ("XML namespaces are considered harmful")?

They release something not called "encoding/xml". They could do what they did to the syscall package. The syscall package, by its nature, can't conform to the 1.0 compatibility promise Go itself maintains, because it changes outside of the scope of the Go project. So they froze the syscall package at some point, and then offered one in the golang.org/x/ namespace at https://pkg.go.dev/golang.org/x/sys . I would again…

There is no good reason for the standard library to include a SAML-safe XML, which is its own huge project, and which is useful only for that one standard. SAML implementations should include their own purpose-built, defensively-written XMLs.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#22

Post by chrsig »

It's worth noting that the go1 compatibility promise[1] allows for breaking compatibility in the name of security issues:

>Security. A security issue in the specification or implementation may come to light whose resolution requires breaking compatibility. We reserve the right to address such security issues.

If the go team decides that this issue is worth a breaking change is another question entirely.

[1]https://golang.org/doc/go1compat

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#23

Post by masklinn »

jerf wrote: post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

> I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they could be. In an XML document using XML namespaces, "ns1:tagname" may not equal "ns1:tagname", and "ns1:tagname" can be equal to "ns2:tagname". This breaks people's mental models of how XML works, and correspondingly, breaks people's code that manipulates XML.

That right there is why I like Clark's notation (despite its unholy verbosity), which I learned of because that's how ElementTree manipulates namespaces: in Clark's notation, the document is conceptually
Which is unambiguous. But as you note adds challenges for round-trip equality (in fact ElementTree doesn't maintain that, it simply discards the namespace prefixes on parsing, which I have seen outright break supposedly XML parsers which were really hard-coded for specific namespace prefixes).

lxml does round-trip prefixes (though it still doesn't round-trip documents) by including a namespace map on each element.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#24

Post by lyxsus »

jerf wrote: post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

> This breaks people's mental models of how XML works, and correspondingly, breaks people's code that manipulates XML.

Because they usually have incorrect mental model. Blaming namespaces for name ambiguity would be the same as blaming the code "x = a + b" because "a" and "b" could be defined differently.

Namespace prefixes are absolutely irrelevant, they only exists for your convenience.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#26

Post by tptacek »

lyxsus wrote: post #24
jerf wrote: post #6

Anyone have examples of XML that can be mutated? My guess is that it wouldn't take much. I expect that a similar problem will be found in many other libraries, if the XML was publicized. XML namespaces made a critical... "mistake" is probably too strong, but "design choice that deviated too far from people's mental model" is about right... that has prevented them from being anywhere near as useful or safe as they cou…

> This breaks people's mental models of how XML works, and correspondingly, breaks people's code that manipulates XML. Because they usually have incorrect mental model. Blaming namespaces for name ambiguity would be the same as blaming the code "x = a + b" because "a" and "b" could be defined differently. Namespace prefixes are absolutely irrelevant, they only exists for your convenience.

That doesn't seem accurate at all. It would be the case if there was some deterministic abbreviation from URL namespace qualifiers down to namespace prefixes, but there is not; instead, they are template variables, which can be shuffled throughout an XML document, requiring security software to constantly and reliably keep track of the value of the variable at multiple points. People sign URLs and JSON documents all the time with schemes that don't have this goofy property.

There's a similar problem with XML entity references, which have been happily breaking enterprise security for over a decade, because nobody has a good mental model of how entities in XML documents actually behave.

It seems fair at this point to blame the standard.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#27

Post by llimllib »

tptacek wrote: post #11
ben509 wrote: post #5

Earlier quoted context omitted.

It's really nuts trying to implement something like SAML in XML. The standard is a security minefield.

What makes it worse is that XMLDSIG is exponentially more complicated. Most of the ecosystem literally shells out to libxmlsec1 and assumes it does the right thing. DSIG is a batshit standard that attempts to support arbitrary combinations of signed and unsigned parts in a single document, tied together with a DOM-like scheme, passed through a canonicalizing transformation that has itself broken SAML before. It's a f…

Also it's my experience that nobody follows the standard - producing valid SAML isn't enough, you need to produce the exact SAML your consumer expects or receivers will reject it. (The context here was passing users off from healthcare.gov to issuers)

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#28

Post by tptacek »

llimllib wrote: post #27
tptacek wrote: post #11

Earlier quoted context omitted.

What makes it worse is that XMLDSIG is exponentially more complicated. Most of the ecosystem literally shells out to libxmlsec1 and assumes it does the right thing. DSIG is a batshit standard that attempts to support arbitrary combinations of signed and unsigned parts in a single document, tied together with a DOM-like scheme, passed through a canonicalizing transformation that has itself broken SAML before. It's a f…

Also it's my experience that nobody follows the standard - producing valid SAML isn't enough, you need to produce the exact SAML your consumer expects or receivers will reject it. (The context here was passing users off from healthcare.gov to issuers)

What makes it worse is that there are practical reasons to implement that way; I've done so for clients, because of bugs found in other SAML parsers that we couldn't leave people susceptible to. One of the material things you can do to lock down a SAML implementation is to accept only the pattern of XML tokens you expect from mainstream IdPs, and then wait for people to complain.

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#29

Post by bawolff »

tptacek wrote: post #11
ben509 wrote: post #5

Earlier quoted context omitted.

It's really nuts trying to implement something like SAML in XML. The standard is a security minefield.

What makes it worse is that XMLDSIG is exponentially more complicated. Most of the ecosystem literally shells out to libxmlsec1 and assumes it does the right thing. DSIG is a batshit standard that attempts to support arbitrary combinations of signed and unsigned parts in a single document, tied together with a DOM-like scheme, passed through a canonicalizing transformation that has itself broken SAML before. It's a f…

Not to mention that libxmlsec1 has some insane insecure defaults that are effectively undocumented.

(I'd go into more details, but i literally just sent a security report yesterday to a saml library for using it wrong, so i guess i shouldn't post publicly about it until they fix)

Re: Coordinated disclosure of XML roundtrip vulnerabilities in Go’s standard library

#30

Post by llimllib »

tptacek wrote: post #28
llimllib wrote: post #27

Earlier quoted context omitted.

Also it's my experience that nobody follows the standard - producing valid SAML isn't enough, you need to produce the exact SAML your consumer expects or receivers will reject it. (The context here was passing users off from healthcare.gov to issuers)

What makes it worse is that there are practical reasons to implement that way; I've done so for clients, because of bugs found in other SAML parsers that we couldn't leave people susceptible to. One of the material things you can do to lock down a SAML implementation is to accept only the pattern of XML tokens you expect from mainstream IdPs, and then wait for people to complain.

I'm so happy I no longer need to work with it. I wrote a manifesto on how it (doesn't) work for the person that replaced me on that project, and it was long, detailed, and angry
Post reply on HN