Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Signing data structures the wrong way

blog.foks.pub

Post reply on HN
Print view View original

21–30 of 59 posts

Re: Signing data structures the wrong way

#21

Post by maxtaco »

Muromec wrote: post #11

So another lesson had been relearned from asn.1. I'm proud of working in this industry again! Next we will figure out to always put versions into the data too

I would say two problems with the asn.1 approach are: (1) it seems like too much cognitive overload for the OIDs to have semantic meaning, and it invites accidental reuse; I think it matters way more that the OIDs are unique, which randomness gets you without much effort; and (2) the OIDs aren't always serialized first, they are allowed to be inside the message, and there are failures that have resulted (https://nvd.nist.gov/vuln/detail/cve-2022-24771, https://nvd.nist.gov/vuln/detail/CVE-2025-12816)

(edit on where the OIDs can be, and added another CVE)

Re: Signing data structures the wrong way

#22

Post by jeffrallen »

lokar wrote: post #8
formerly_proven wrote: post #4

This article claims that these are somewhat open questions, but they're not and have not been for a long time. #1 You sign a blob and you don't touch it before verifying the signature (aka "The Cryptographic Doom Principle") #2 Signatures are bound to a context which is _not_ transmitted but used for deriving the key or mixed into the MAC or what have you. This is called the Horton principle. It ensures that signer/v…

What if (and this is perhaps to big an if), you only ever serialize and de-serialize with code generated from the IDL, which always checks the magic numbers (returning a typed object(?

It's a big if because the threat model normally includes "bad guys can forge messages". Which means that the input is untrusted and you want to generate your own domain separation bytes for the hash function, not let your attacker choose them.

Re: Signing data structures the wrong way

#23

Post by tennysont »

formerly_proven wrote: post #4

This article claims that these are somewhat open questions, but they're not and have not been for a long time. #1 You sign a blob and you don't touch it before verifying the signature (aka "The Cryptographic Doom Principle") #2 Signatures are bound to a context which is _not_ transmitted but used for deriving the key or mixed into the MAC or what have you. This is called the Horton principle. It ensures that signer/v…

Hmmmm. I agree that an ad-hoc implementation with protobufs can go wrong. But presumably, 1 canonical encoding for the private key constitutes the Horton principle?

It seems like Horton Principle just says "all messages have ≤1 meaning". If a message signed by key X must be parsed using the canonical encoding, then aren't we done?

There is still room for danger. e.g., You send `GetUserPermissionLevel(user:"Alice")` and server responds with `UserNicknameIs(user:"Alice", value:"admin")`. If you fail to check the message type, you might get tricked.

Maybe it's nice if it was mathematically impossible to validate the signature without first providing your assumptions. e.g., The subroutine to validate message `UserNicknameIs(user:"Alice", value:"admin")` requires `ServerKey × ExpectedMessageType`. But "ExpectedMessageType" isn't the only assumption being made, is it?

You might get back `UserPermissionLevel(user:"Bob", value:"admin")` or `UserPermissionLevel(user:"Alice", value:"admin", timestamp:"")`. Will we expect the MAC to somehow accept a "user" value? And then what do we do about "timestamp"?

Maybe we implement `ClientMessage(msgUuid: UUID, requestData:...)` and `ServerResponse(clientMsgUuid: UUID, responseData:...)`, but now the UUID is a secret, vulnerable to MITM attack unless data is encrypted.

It seems like you simply must write validation code to ensure that you don't misinterpret the message that is signed. There simply isn't any magic bullet. Having multiple interpretations for a sequence of bytes is a non-starter (addressed in the post). But once you have a single interpretation for a sequence of bytes, isn't it up to the developer to define a schema + validation logic that supports their use case? Maybe there are good off-the-shelf patterns, but--again--no magic bullets?

Re: Signing data structures the wrong way

#24

Post by tennysont »

nightpool wrote: post #9
lokar wrote: post #7

Earlier quoted context omitted.

No, I'm pretty sure they are saying you need to transmit it

No, they propose just concatenating it with the data received from the network > it makes a concatenation of the domain separator (@0x92880d38b74de9fb) and the serialization of the object, and then feeds the byte stream into the signing primitive. Similarly, verification of an object verifies this same reconstructed concatenation against the supplied signature. > Note that the domain separator does not appear in the…

You are, of course, right. And this distinction is important for this chain of comments.

Though, in fairness, that is /kind of/ like transmitting it---in the sense that it impacts the message that is returned. It's more akin to sending a checksum of the magic number, rather than the magic number itself. But conceptually, that is just an optimization. The desire is for the client to ensure the server is using the same magic number, we just so happen to be able to overload the signature to encode this data without increasing the message size.

Re: Signing data structures the wrong way

#26

Post by tennysont »

logicallee wrote: post #12

along the same lines, did you know that you can get an authenticated email that the listed sender never sent to you? If the third party can get a server to send it to themselves (for example Google forms will send them an email with the contents that they want) they can then forward it to you while spoofing the from: field as Google.com in this example, and it will appear in your inbox from the "sender" (Google.com)…

I asked about this on the PGP mailing list at one point, and I think I was told that the best solution is to start emails with "Hi ," which seems like a funny low-tech solution to a (sad) problem.

Re: Signing data structures the wrong way

#28

Post by maxtaco »

cogman10 wrote: post #25

Why not digest the type as part of the hash? This avoids the problem in the article and keeps the transmission size small.

It should be possible to change the name of the type, and this happens often in practice. But type renames shouldn't break preexisting signatures. In this scheme you are free change the type name, and preexisting signatures still verify with new code -- of course as long as you never change the domain separator, which you never should do. Also you'd need to worry about two different projects reusing the same type name. Lastly, the transmission size in this scheme remains unaffected since the domain separators do not appear in the serialized data. Rather, both sides agree on it via the protocol specification.

Re: Signing data structures the wrong way

#29

Post by tennysont »

cogman10 wrote: post #25

Why not digest the type as part of the hash? This avoids the problem in the article and keeps the transmission size small.

They use a magic number, rather than a digest derived from the schema[1], but otherwise they do as you suggest. The magic number is given to the signing function (sender side) and the validation function (receiver side) but does not increase the size of the transmitted message.

[1]

I think that's what you mean by digest, but maybe you just mean `type` = `magic number`

Re: Signing data structures the wrong way

#30

Post by kccqzy »

majormajor wrote: post #5
Retr0id wrote: post #2

Putting domain separators in the IDL is interesting but you can also avoid the problem by putting the domain separators in-band (e.g. in some kind of "type" field that is always present). Tangentially, depending on what your input and data model look like, canonicalisation takes O(nlogn) time (i.e. the cost of sorting your fields). Here I describe an alternative approach that produces deterministic hashes without a d…

I think a lot of people assume that the "name" of the type, for protos, will be preserved somewhere in the output such that a TreeRoot couldn't be re-used as a KeyRevoke. It makes sense that it isn't - you generally don't want to send that name every time - but it's non-obvious to people with a object-oriented-language background who just think "ah, different types are obviously different types." The serialization co…

This is just a mismatch between nominal typing and structural typing. Protobuf is basically structural typing. You can serialize a message defined with one schema and deserialize the result to a message with a different schema if the two schemata are compatible enough. Almost all normal programming languages use nominal typing. If you have `struct A {int a; int b};` it is distinct from `struct B {int a; int b};`.
Post reply on HN