41–50 of 51 posts
We built a tamper evident log system for https content. The beta is over here: https://www.transparencylog.com/ Or try out the CLI tool https://github.com/transparencylog/tl#installation
github.com/arianvp/kpop.sh
I'm happy to see my idea is not novel. That means I was on the right track :P
Earlier quoted context omitted.
Every time I've seen syslog in the wild, it's been the old "BSD" version (RFC 3164), not the newer RFC 5424 version. And pretty much any newer format, even something basic like JSON-lines over TCP, is going to be nicer to work with, IMO. (Though I admit, I have no idea how I'd even begin to start signing messages with JSON-lines.)
Got to believe most JSON parsers can be set to permissive mode to allow comments, and then you could just append `//signature` on to the end of each JSON line?
2009-05-03T14:00:39.529966+02:00 host.example.org syslogd 2138 - [ssign VER="0111" RSID="1" SG="0" SPRI="0" GBC="2" FMN="1" CNT="7" HB="K6wzcombEvKJ+UTMcn9bPryAeaU= zrkDcIeaDluypaPCY8WWzwHpPok= zgrWOdpx16ADc7UmckyIFY53icE= XfopJ+S8/hODapiBBCgVQaLqBKg= J67gKMFl/OauTC20ibbydwIlJC8= M5GziVgB6KPY3ERU1HXdSi2vtdw= Wxd/lU7uG/ipEYT9xeqnsfohyH0=" SIGN="AKBbX4J7QkrwuwdbV7Taujk2lvOf8gCgC62We1QYfnrNHz7FzAvdySuMyfM="]
You're trying to re-invent the wheel. Look at the prior art.
I dont particularly agree with an obvious use of a merkle tree being patented in this way - but it is. Probably worthy of further research if you plan to use this.
Couldn't we use, you know, git for the many applications that require a verifiable history of commits but don't need to be super fast or real-time?
" Don't be snarky. " " Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. " https://news.ycombinator.com/newsguidelines.html You may have a good point, but for it to be a good comment you would need to drop the snark and engage more specifically with the details of the article. The odds, for example, that the authors don't know about or have thought…
philips: please accept my apologies too. The last thing I want to be is snarky.
Couldn't we use, you know, git for the many applications that require a verifiable history of commits but don't need to be super fast or real-time?
Git has an issue that branch heads are mutable, and unsigned, so two repos can disagree about what is head with the same commits in them. Using a transparency log to record changes in heads is like forcing no fast forwards and signing each change in heads.
Earlier quoted context omitted.
" Don't be snarky. " " Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. " https://news.ycombinator.com/newsguidelines.html You may have a good point, but for it to be a good comment you would need to drop the snark and engage more specifically with the details of the article. The odds, for example, that the authors don't know about or have thought…
dang: I did not mean for my comment to be snarky! I also did not mean it to be dismissive! I meant it as an honest, obvious question one should ask. But in hindsight, in the light of morning, my comment sure looks snarky, you're right. I wrote it quickly last night. Argh. Sorry. And thank you for taking the time to point it out. philips: please accept my apologies too. The last thing I want to be is snarky.
Earlier quoted context omitted.
> This isn't obviously anything new. Conceptually, maybe not, given the history of Merkle trees and hash chains. But examples of production code using these ideas are few and far between. That's still valuable.
Sorry, speaking loosely. I meant that this doesn't seem to be an announcement of a new open source repo. Certificate transparency itself has been in production for a while. Apparently since 2013 and required for all certificates since 2018, according to Wikipedia. Depending on how you define it, everyone who publishes source code using git is already using these ideas. It seems they are all Merkle trees? I would gues…
I agree that the presentation is a bit confusing. It feels as though the audience are in directorship/VP roles, but the .dev TLD is usually oriented towards folks in the trenches. I think the idea was to distinguish the general idea (tamper-evident logs) from the specific implementation (Trillian). I already knew about Trillian and found myself unsure as to whether it was about Trillian or not.
> Depending on how you define it, everyone who publishes source code using git is already using these ideas. It seems they are all Merkle trees?
Merkle trees have been around for a smidge over 40 years. I don't think their claim is to have invented a new data structure, but rather that they provide a smooth way to apply Merkle trees to log-structured and map-structured problems.
> If you can download the whole dataset (like with git) and you are making manual edits and sending pull requests, then Dolt's approach sounds more user-friendly? You get a change history, and if upstream rebases then you will see it.
The orientation to logs here suggests that they're not very interested in forking or merging. Added to which, there's no sense in which the subject of discourse is snapshots to which one can apply deltas to emit a successor snapshot.
I believe the idea is to ensure that a particular sequence of events occurred, that it occurred in a particular order, that no entries were inserted, deleted or modified without being detected. In git I believe the closest analogy would be the reflog.
This seems to be the same as guardtimes (patented) hash-calendar? https://patents.google.com/patent/US8312528 I dont particularly agree with an obvious use of a merkle tree being patented in this way - but it is. Probably worthy of further research if you plan to use this.
Put another way: this patent may cover some specific applications of Merkle trees to natural numbers, but doesn't cover all sequential data structures. For most purposes "happened-at" or "happens-before" relations are plenty useful and don't require sequence numbers to establish the relation.
Sorry I read the page and I found it very vague. Pretty graphics and lot of white space for not much meat. Basically, looks like the command pattern implemented with signed token for every time you read, you must mutate the 'chain' or array of information. Am I close?
Many thanks for the feedback on the site, perhaps we need to get some more dense technical details either on there directly or adding to the existing docs on the github repo. Command pattern is definitely one pattern we've seen these tamper-evident logs applied to; you can easily imagine a system where, say, config update "commands" are packaged off and written to a TE log, and then workers apply those commands to a…
Tip: call it LogOps and engage in an exercise-yard shiv-fight with GitOps folks.
I prefer to think of both as examples of WAL or intent logs. It's a useful pattern and it's nice to have a narrowly-focused tool for it.
Earlier quoted context omitted.
Sorry, speaking loosely. I meant that this doesn't seem to be an announcement of a new open source repo. Certificate transparency itself has been in production for a while. Apparently since 2013 and required for all certificates since 2018, according to Wikipedia. Depending on how you define it, everyone who publishes source code using git is already using these ideas. It seems they are all Merkle trees? I would gues…
> I meant that this doesn't seem to be an announcement of a new open source repo. I agree that the presentation is a bit confusing. It feels as though the audience are in directorship/VP roles, but the .dev TLD is usually oriented towards folks in the trenches. I think the idea was to distinguish the general idea (tamper-evident logs) from the specific implementation (Trillian). I already knew about Trillian and foun…
A log tells you what happened. Or really, what events were successfully recorded, which might be different. But it's also common to want to know the result of a sequence of transactions on system state, and to be able to reconstruct that state at any point in time. (For example, there is a debate about when it makes sense to use Kafka as a database.)
So, someone deciding whether to use this might consider whether they just want to publish a tamper-evident log, or whether they are publishing something that looks more like a dataset with a temper-evident change history. A big factor in deciding that will be how large this dataset is likely to be and whether it's feasible and useful for the reader to download a complete copy of it, as we do with git repos.
You also have to consider the criteria for accepting events. This determines what data you need to query before accepting a transaction. The easy case is when all well-formed events are accepted, but for some problems, it makes sense to want more consistency. The reader might want to check consistency too.