31–40 of 51 posts
Couldn't we use, you know, git for the many applications that require a verifiable history of commits but don't need to be super fast or real-time?
I don’t understand one part though. Because checking for log integrity is fairly obvious and easy to understand (merkle trees).
However, I don’t see how this creates a log that can’t be tampered with.
If you change a record in a log, you have to change all records following it, in order to preserve integrity of hashes, but if that record has been stored recently, that wouldn’t be terribly hard. So where’s the magic that make this impossible?
With bitcoin the magic is that mining hashes for a transaction block takes a lot of resources and the longest chain always wins.
Here I don’t see anything of this kind.
I'm definitely a layman as far as crypto is concerned, so take the above with a grain of salt. Looking for somebody more knowledgable about the topic to clear my confusion.
EDIT: Looking at https://transparency.dev/verifiable-data-structures/ there's the part `Clients must store a copy of tree head hashes`, so that partly solves the problem I suppose. But aren't the clients (who I understand have to consume all produced records in order to continuously recalculate the root hash) then ripe for tampering?
I’ve said it for years, but straightforward approaches to signing logs and database dumps takes care of most problems blockchains are supposed to solve.
They are related, but not quite the same.
Looks neat. I don’t understand one part though. Because checking for log integrity is fairly obvious and easy to understand (merkle trees). However, I don’t see how this creates a log that can’t be tampered with. If you change a record in a log, you have to change all records following it, in order to preserve integrity of hashes, but if that record has been stored recently, that wouldn’t be terribly hard. So where’s…
If it's append only, you should be able to find the old head in the log. Not sure if that is what it's going for though. I'm still a bit confused by this myself.
If anyone wants to see how someone built this in production without any of the "we're building a movement, we have a fancy landing page" hype: Secure logging with syslog-ng Forward integrity and confidentiality of system logs @ FOSDEM 2020[0]. I'm starting to become really suspicious of things being pushed as standards/new approaches/things-that-should-be-rfcs-but-have-a-marketing-budget. As other posts have pointed…
Looks neat. I don’t understand one part though. Because checking for log integrity is fairly obvious and easy to understand (merkle trees). However, I don’t see how this creates a log that can’t be tampered with. If you change a record in a log, you have to change all records following it, in order to preserve integrity of hashes, but if that record has been stored recently, that wouldn’t be terribly hard. So where’s…
As you say, Merkle trees are fairly straight-forward, but beyond integrity checking a file they have some other nice properties - as the "operator" of a Merkle tree based log, you can efficiently [O(log N)] convince an observer that: - you've only grown the log by appending new entries since the last time they looked ("consistency proof") - a given entry they hold is indeed included in the log ("inclusion proof")
if you change an entry in the log (or claim something is in the log when it's not) it becomes impossible for you to prove both of those properties (unless you're powerful enough to also break what's called the "second pre-image resistance" of the cryptographic hash you're using).
Note that for these two properties, an observer doesn't need to download or inspect the full contents of the log, just a handful of hashes in the Merkle tree which they can then use to recreate the "tree head hash" you mentioned. These "tree heads" are the state they store and are just a few hundred bytes.
There are generally a few types of log client: - one type is putting things into the log - one type are wanting to use what's stored in the log to make a decision of some kind ("should I trust this site", "should I install this binary", "am I seeing the true history of events", etc.) and they won't trust that data unless they are convinced it is in the log - another can verify the accuracy/validity/goodness of things in the log (e.g. in CT it's the domain owner who is in the unique position of saying "yes, I asked for this cert to be created", for binaries it might be "I'm the authorised creator of these binaries and, yup, that was me" or "I'm a respectable malware scanning company, and this binary doesn't match any known signatures", "I'm an auditor and these transactions are complete and correct", etc. - you can probably think of many more examples!)
In this light, you can think of a tamper-evident log as being a sort of "reliable transport" to connect these different types of clients together, and importantly, where they all can prove to each other (using those light weight tree heads + consistency/inclusion proofs) that they're seeing the same set of data in the log - or, if they can't prove it - they have the cryptographic evidence of operator of the log operator having done something unexpected.
If you have a spare few mins, have a look at the Claimant Model docs here: https://github.com/google/trillian/tree/master/docs/claimant... which talks in a bit more detail about this "transport" idea and the concept of discoverability which it brings.
Hope that's helpful/interesting!
[Disclaimer: I work on CT, Trillian, and a bunch of other related projects.]
This isn't obviously anything new. From the github repo, it looks like Google open-sourced the underlying implementation for certificate transparency a while back. [1] Any DNS registry will need something similar to this to participate. While in theory, anyone publishing public information might find something like this useful, it's not clear there's much demand for it outside DNS. Something like Dolt [2] seems like…
I'm seeing this as an alternative to blockchain hype for cases where there isn't a double-spend problem (which is most of the non-public-access-currency use cases people are excited about) or where you can't assume that the network controls more computational power than any attacker (which is basically all of them). Just about any "permissioned blockchain" use case is better off using this. Basically, they're pitchin…
We're looking in more depth at other use cases, developing a better understanding of which types of problems this might be useful for, and ways to reason about the properties you want/get from using systems like this (e.g. https://github.com/google/trillian/tree/master/docs/claimant...)
[Disclaimer: I work on CT, Trillian, and some other related projects]
Looks neat. I don’t understand one part though. Because checking for log integrity is fairly obvious and easy to understand (merkle trees). However, I don’t see how this creates a log that can’t be tampered with. If you change a record in a log, you have to change all records following it, in order to preserve integrity of hashes, but if that record has been stored recently, that wouldn’t be terribly hard. So where’s…
I prefer to use the term "tamper-evident" when describing these things - you can totally fiddle with logs, but the property you likely want is that this can be detected. As you say, Merkle trees are fairly straight-forward, but beyond integrity checking a file they have some other nice properties - as the "operator" of a Merkle tree based log, you can efficiently [O(log N)] convince an observer that: - you've only gr…
Especially this part is enlightening: In this light, you can think of a tamper-evident log as being a sort of "reliable transport" to connect these different types of clients together [...]
As it shows why my observation about possible tampering with clients isn't all that important.
Sorry I read the page and I found it very vague. Pretty graphics and lot of white space for not much meat. Basically, looks like the command pattern implemented with signed token for every time you read, you must mutate the 'chain' or array of information. Am I close?
Command pattern is definitely one pattern we've seen these tamper-evident logs applied to; you can easily imagine a system where, say, config update "commands" are packaged off and written to a TE log, and then workers apply those commands to a system if and only if they are convinced that the commands were properly logged. If the admins/command-workers/auditors etc. can all prove they're looking at the same set of logged commands (which is the property that these tamper-evident logs are intended to provide), then you have a reliable and tamper-evident audit trail of all changes to your system.
There are a bunch of other scenarios we think these things might be useful for too, and we're trying to explore (and help others explore) these and provide mechanisms for reasoning about the properties of such systems.
[Disclaimer: I work on CT, Trillian, and other related projects]
RFC 5848, "Signed Syslog Messages": > This document describes a mechanism to add origin authentication, message integrity, replay resistance, message sequencing, and detection of missing messages to the transmitted syslog messages. This specification is intended to be used in conjunction with the work defined in RFC 5424, "The Syslog Protocol". * https://tools.ietf.org/html/rfc5848
Every time I've seen syslog in the wild, it's been the old "BSD" version (RFC 3164), not the newer RFC 5424 version. And pretty much any newer format, even something basic like JSON-lines over TCP, is going to be nicer to work with, IMO. (Though I admit, I have no idea how I'd even begin to start signing messages with JSON-lines.)
> [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"][examplePriority@32473 class="high"]