The blog post provides a certificate chain that validates in OpenSSL but not in Go.
The reason it doesn't validate in Go is that the Subject field in the CA certificate uses a different string encoding than the Issuer field in the leaf certificate, so the fields are not byte-for-byte equal.
Go requires the Issuer and Subject to be byte-for-byte equal. This was permitted by older specs, but RFC 5280 changed the rules to require the use of RFC 4518 (LDAP stringprep) for comparing strings. This turned a simple memcmp into a complicated algorithm that requires implementing Unicode normalization, for virtually zero benefit. That's the last thing you want in your security-critical certificate verifier, so Go quite sensibly chose to follow the older specs in this regard. The CA/Browser Forum's Baseline Requirements also mandate byte-for-byte equality, so Go's behavior won't cause publicly-trusted certificates to be incorrectly rejected.
Note that LDAP stringprep is so complicated that OpenSSL doesn't even try to implement it properly and uses an approximation instead. So you would also be able to "fool" OpenSSL into rejecting certificate chains that RFC 5280 says are valid.
The blog post says that this is an "ongoing debate" in the Go project but I don't think that's accurate. I'd be shocked if they ever changed this behavior, given that crypto/x509 targets publicly-trusted certificates and the current behavior is so much simpler.
Something I noticed decades ago is that some small, innocuous features can drag with them giant ecosystems of software.
I first noticed this when I had to implement a C++ client for a custom RPC protocol and the dev “on the other end of the wire” added one new “convenience” in the data types supported… which would have required me to include the entire Java runtime in my client!
All protocol specs are vulnerable to this effect where it’s all too easy to require clients to include half a dozen different regex engines, three byte code virtual machines, and most of LLVM for good measure.
We should require 2 different implementation, each in different enough (so no C/C++ pair) language with each specification.
Because that way we not only get rid of language's smell in how stuff is implemented, but also the act of implementing the spec will quickly show any cases where it looked simple in spec but turns out to be mess implementation wise.
Too much work ? Well, make your spec be tighter and simpler before you burden the rest of programming community with implementing it
This is uninteresting. CAs are well aware that they have to encode the subject DN and issuer DN identically to maximize interoperability. There are several implementations that require that.
If we were to make a new version of the spec for X.509 certificates, I would hope that we would eliminate all the non-UTF8 encodings so that this would be a non-issue.
To be clear, the differential here occurs because OpenSSL does the wrong thing. Go is correct to fail closed here, and it’s very hard to imagine a setting in which Go failing closed is a relevant security differential.
Just to be clear, OpenSSL isn't doing the wrong thing, based on the description in the blog post. The specification allows and even requires behavior similar to that.
They’re two different fields on different certificates. The issuer on the child needs to match the subject on the parent; every certificate has its own subject and issuer.
Perhaps I overlooked something, but which part of this involved "fooling Go"? I would usually not call it "fooling" something to trigger a not-strictly-required rejection of a dubious trait, especially when best practice says to avoid that trait.
The title of this post is wildly misleading: Go’s behavior is correct here. But even if it wasn’t, failing closed isn’t generally a way one fools a X.509 path validator.
Source: having had the displeasure of implementing an RFC 5280 and CABF-conforming path validator.
This is a great example of how format complexity creates security problems.
The same thing shows up in PDFs: two tools can read the same file and disagree about what's actually in it because they interpret the structure differently.
X.509, PDFs, Office docs. Whenever multiple implementations are making judgment calls on the same bytes, those edge cases become very interesting from a security perspective.
This is what happens when kids today are unaware of history. This was a known problem 30+ years ago, and the Go kids have just rediscovered it for themselves. The most extreme case of this madness was imagining you could re-encode certificates into a DER blob from their stored components and the signature would still validate, something that OER (from memory) guys are now trying to do.
The rules for DNs are "there is only one encoding rule and that is memcpy(); there is only one matching rule and that is memcmp()". Given that Go has fallen into the decades-old trap of trying to re-encode strings, it's bound to be vulnerable to any number of other issues like evading excludedSubtrees through string-encoding tricks.
Between this and the IPv6 zone identifier issue, it feels like there's a bit of a trend of commenters more or less assuming Go is doing the wrong thing when it's actually following the standards/best practices more correctly than average. I wonder where this reputation came from.
That's a different point, this particular thread is actually about a behavior that Chrome/OpenSSL/etc. have that is actually somewhat undesirable due to being complex and error prone.
Simply following Chrome/OpenSSL/etc. yields the best compatibility, but the Go behavior is much simpler in this very security critical path, and I'm not sure if it really occurs on the web. It seems most people are reporting it for other uses of X.509 like TPM. You can certainly see the argument in favor of Go's vastly simpler approach.
Go has a lot of trade-offs like this. If you can't decode from or encode into UTF-8 a given filename, Go's high level I/O APIs can't deal with it - depending on your exact point of view, the kinds of software you write and certainly some level of taste, you may see this as horrible, or you may see it as a totally reasonable trade-off. Some programs can go ahead and say "Sorry, we only support UTF-8 filenames" and some really can't.
But when being rational and reasoned it's really challenging to make the argument that Go's choices are bad due to inexperience from the language designers or standard library authors or general poor attention to detail. Calling the developers "Go kids" is already amusing considering it is very famously a project originally created by Rob Pike, Ken Thompson and Robert Griesemer - not foolish young developers who are likely to repeat mistakes of the past by way of ignorance. Of course, that doesn't mean say, the TLS stack, deserves to carry some special weight just because the language was designed by computer science stalwarts, but I've dug into the Go TLS stack a fair bit in my leisure and to me personally it definitely does not feel like the work of people who are ignorant or foolish.
I may be over-indexing on two recent comments here, but it does sort of feel like there's an undercurrent of this, and it bums me out because I particularly like Go for the exact opposite reason, I really think a lot of the tradeoffs, love them or hate them, are very well thought out.
edit: For posterity sake, it is worth noting that it was wrong for me to say "Chrome/OpenSSL/etc." as Chrome actually has a similar behavior to Go here.
From the article, it doesn't seem like Go is trying to re-encode strings? Go is saying (correctly, IMO) that a UTF8String field in the Issuer is not the same as a PrintableString field in the Subject.
Ah, you're right, I was a bit confused by the bouncing back and forth between Go and OpenSSL and the title, "Fooling Go's X.509" when in fact on re-read Go appears to be doing the right thing and using a strict compare while OpenSSL uses the open-to-manipulation compare.
The blog post provides a certificate chain that validates in OpenSSL but not in Go.
The reason it doesn't validate in Go is that the Subject field in the CA certificate uses a different string encoding than the Issuer field in the leaf certificate, so the fields are not byte-for-byte equal.
Go requires the Issuer and Subject to be byte-for-byte equal. This was permitted by older specs, but RFC 5280 changed the rules to require the use of RFC 4518 (LDAP stringprep) for comparing strings. This turned a simple memcmp into a complicated algorithm that requires implementing Unicode normalization, for virtually zero benefit. That's the last thing you want in your security-critical certificate verifier, so Go quite sensibly chose to follow the older specs in this regard. The CA/Browser Forum's Baseline Requirements also mandate byte-for-byte equality, so Go's behavior won't cause publicly-trusted certificates to be incorrectly rejected.
Note that LDAP stringprep is so complicated that OpenSSL doesn't even try to implement it properly and uses an approximation instead. So you would also be able to "fool" OpenSSL into rejecting certificate chains that RFC 5280 says are valid.
The blog post says that this is an "ongoing debate" in the Go project but I don't think that's accurate. I'd be shocked if they ever changed this behavior, given that crypto/x509 targets publicly-trusted certificates and the current behavior is so much simpler.
Something I noticed decades ago is that some small, innocuous features can drag with them giant ecosystems of software.
I first noticed this when I had to implement a C++ client for a custom RPC protocol and the dev “on the other end of the wire” added one new “convenience” in the data types supported… which would have required me to include the entire Java runtime in my client!
All protocol specs are vulnerable to this effect where it’s all too easy to require clients to include half a dozen different regex engines, three byte code virtual machines, and most of LLVM for good measure.
any specification eventually grows to encompass all features of its original implementation language.
We should require 2 different implementation, each in different enough (so no C/C++ pair) language with each specification.
Because that way we not only get rid of language's smell in how stuff is implemented, but also the act of implementing the spec will quickly show any cases where it looked simple in spec but turns out to be mess implementation wise.
Too much work ? Well, make your spec be tighter and simpler before you burden the rest of programming community with implementing it
I feel like basically all the X509 threads on HN should basically be locked until after you write your first comment on them.
Aw, thanks :-)
[flagged]
I don't think that's an ongoing debate looks to me that it ended in 2019 https://github.com/golang/go/issues/31440#issuecomment-53724...
This is uninteresting. CAs are well aware that they have to encode the subject DN and issuer DN identically to maximize interoperability. There are several implementations that require that.
If we were to make a new version of the spec for X.509 certificates, I would hope that we would eliminate all the non-UTF8 encodings so that this would be a non-issue.
That is like saying you can't get a virus on your computer because Facebook doesn't allow viruses to be posted to the internet.
Differential parsing is a whole class of security bugs and they matter a lot. Take a look at HTTP Request Smuggling for examples.
Also, I am pretty sure there are more non-web x509 certificates out there than all the "browser trusted CAs" combined have signed. :)
To be clear, the differential here occurs because OpenSSL does the wrong thing. Go is correct to fail closed here, and it’s very hard to imagine a setting in which Go failing closed is a relevant security differential.
Just to be clear, OpenSSL isn't doing the wrong thing, based on the description in the blog post. The specification allows and even requires behavior similar to that.
If differential parsing of X.509 certificates is a material security concern for something, then that's a bug in that thing.
Even outside of the web PKI, the requirement for identical encoding is also often found in certificate policies used within closed communities.
uninteresting to you, i learned something new.
if 2 fields are identical, why aren't they 1 field?
They’re two different fields on different certificates. The issuer on the child needs to match the subject on the parent; every certificate has its own subject and issuer.
Perhaps I overlooked something, but which part of this involved "fooling Go"? I would usually not call it "fooling" something to trigger a not-strictly-required rejection of a dubious trait, especially when best practice says to avoid that trait.
The title of this post is wildly misleading: Go’s behavior is correct here. But even if it wasn’t, failing closed isn’t generally a way one fools a X.509 path validator.
Source: having had the displeasure of implementing an RFC 5280 and CABF-conforming path validator.
This is a great example of how format complexity creates security problems.
The same thing shows up in PDFs: two tools can read the same file and disagree about what's actually in it because they interpret the structure differently.
X.509, PDFs, Office docs. Whenever multiple implementations are making judgment calls on the same bytes, those edge cases become very interesting from a security perspective.
This is what happens when kids today are unaware of history. This was a known problem 30+ years ago, and the Go kids have just rediscovered it for themselves. The most extreme case of this madness was imagining you could re-encode certificates into a DER blob from their stored components and the signature would still validate, something that OER (from memory) guys are now trying to do.
The rules for DNs are "there is only one encoding rule and that is memcpy(); there is only one matching rule and that is memcmp()". Given that Go has fallen into the decades-old trap of trying to re-encode strings, it's bound to be vulnerable to any number of other issues like evading excludedSubtrees through string-encoding tricks.
Between this and the IPv6 zone identifier issue, it feels like there's a bit of a trend of commenters more or less assuming Go is doing the wrong thing when it's actually following the standards/best practices more correctly than average. I wonder where this reputation came from.
Most people accessing a site are likely not using Golang and are using Chrome. Thus Chrome is assumed right and Go is the one that's the outlier.
That's a different point, this particular thread is actually about a behavior that Chrome/OpenSSL/etc. have that is actually somewhat undesirable due to being complex and error prone.
Simply following Chrome/OpenSSL/etc. yields the best compatibility, but the Go behavior is much simpler in this very security critical path, and I'm not sure if it really occurs on the web. It seems most people are reporting it for other uses of X.509 like TPM. You can certainly see the argument in favor of Go's vastly simpler approach.
Go has a lot of trade-offs like this. If you can't decode from or encode into UTF-8 a given filename, Go's high level I/O APIs can't deal with it - depending on your exact point of view, the kinds of software you write and certainly some level of taste, you may see this as horrible, or you may see it as a totally reasonable trade-off. Some programs can go ahead and say "Sorry, we only support UTF-8 filenames" and some really can't.
But when being rational and reasoned it's really challenging to make the argument that Go's choices are bad due to inexperience from the language designers or standard library authors or general poor attention to detail. Calling the developers "Go kids" is already amusing considering it is very famously a project originally created by Rob Pike, Ken Thompson and Robert Griesemer - not foolish young developers who are likely to repeat mistakes of the past by way of ignorance. Of course, that doesn't mean say, the TLS stack, deserves to carry some special weight just because the language was designed by computer science stalwarts, but I've dug into the Go TLS stack a fair bit in my leisure and to me personally it definitely does not feel like the work of people who are ignorant or foolish.
I may be over-indexing on two recent comments here, but it does sort of feel like there's an undercurrent of this, and it bums me out because I particularly like Go for the exact opposite reason, I really think a lot of the tradeoffs, love them or hate them, are very well thought out.
edit: For posterity sake, it is worth noting that it was wrong for me to say "Chrome/OpenSSL/etc." as Chrome actually has a similar behavior to Go here.
From the article, it doesn't seem like Go is trying to re-encode strings? Go is saying (correctly, IMO) that a UTF8String field in the Issuer is not the same as a PrintableString field in the Subject.
Ah, you're right, I was a bit confused by the bouncing back and forth between Go and OpenSSL and the title, "Fooling Go's X.509" when in fact on re-read Go appears to be doing the right thing and using a strict compare while OpenSSL uses the open-to-manipulation compare.
Maybe this should be a lesson about the dangers of rolling in hot with a bunch of insults?
The Go "kids" are famous for, among other things, being industry leaders 30 years ago.
Those young whipper snappers with their lifetime of experience!
Them darn youngins[1]!
[1] Ken Thompson (over 80), Rob Pike (around 70), Robert Griesemer (over 60)