171–180 of 258 posts
Earlier quoted context omitted.
Checksumming filesystems and file transfer protocols cover many cases. SCP, rsync, btrfs, and zfs all fix this problem. As for guaranteeing the computed data is correct: I know space systems often have two redundant computers that calculate everything and compare results. It's crazy expensive and power demanding, but it all but solves the problem.
If they have two computers and results differ, how do they decide which one is correct ? ;-)
Earlier quoted context omitted.
> By definition, if a failure occurs, it's because it passed multiple safeguards that were already in place. Having worked on a good bunch of critical system, there aren't multiple safeguards in most hardware. E.g. a multiplication error in a core will not be detected by an external device. Or a bit flip when reading cache, or from a storage device. Very often the only real safeguard is to do the whole computation tw…
> Very often the only real safeguard is to do the whole computation twice on two different hosts. Three different hosts, for quorum, right?
Earlier quoted context omitted.
> By definition, if a failure occurs, it's because it passed multiple safeguards that were already in place. Having worked on a good bunch of critical system, there aren't multiple safeguards in most hardware. E.g. a multiplication error in a core will not be detected by an external device. Or a bit flip when reading cache, or from a storage device. Very often the only real safeguard is to do the whole computation tw…
> Very often the only real safeguard is to do the whole computation twice on two different hosts. Three different hosts, for quorum, right?
The push for crypto without ecc ram is a nonstop horror show. Software under normal circumstances is remarkably resilient to having its memory corrupted. However crypto algorithms are designed so that a single bit flip effectively changes all the bits in a block. If you chain blocks then a single bit flip in one block destroys all the blocks. I've seen companies like msps go out of business because they were doing cr…
If it is caused by a single bitflip you know the block in which that bitflip occurred and can try each bit until you find the right bit. This is an embarrassing parallel problem. Let's say you need to search 1 GB space for a single bit flip. That only requires that you test 8 billion bit flips. Given the merklized nature of most crypto, you will probably be searching a space far smaller than 1 GB.
>Bit flips aren't an act of god you simply need a better computer.
Rather than using hardware ECC, you could implement ECC in software. I think hardware ECC is good idea, but you aren't screwed if you don't use it.
The big threat here is not the occasional random bit flips, but adversary caused targeted bit flips since adversaries can bit flip software state that won't cause detectable failures but will cause hard to detect security failures.
As far as I know, recent Android devices and iPhones have full disk encryption by default, but they do protect the keys against random bit-flipping?
Also, I guess that authentication devices (like YubiKey) are not safe and easy to use at the same time, because the private key inside can be damaged/modified by a cosmic ray, and it's not possible (by design) to make duplicates. So, it's necessary to have multiple of them to compensate, lowering their practicality in the end.
Edit: from the software side, I understand that there are techniques to ensure some level of data safety (checksumming, redudancy, etc), but it thought it was OK to have some random bit-flipping on hard disks (where I found it more frequently), since it could be corrected from software. Now I realize that if the encryption key is randomly changed on RAM, the data can or becomes permanently irrecoverable.
Earlier quoted context omitted.
> Very often the only real safeguard is to do the whole computation twice on two different hosts. Three different hosts, for quorum, right?
Depends on the system. In this case it seems like retries are possible after a failure, so two is sufficient to detect bad data. You need three in real time situations where you don't have the capability to go back and figure it out.
Sometimes doing the same processing twice can be also a way to implement safe(r) rolling updates.
Earlier quoted context omitted.
If a system is critical it should run on multiple machines in multiple locations and "sync with checks" kinda like the oh so hated and totally useless blockchains. Then if such a bit-flip would occur it would never occur on all machines at the same time in the same data. And on top of that you could easy make the system fix itself if something like that happens (simply assume the majority of nodes didn't have the bit…
I haven't thought about whether an actual blockchain is really the best solution, but the redundancy argument is legitimate. We've been doing it for decades in other systems where an unnoticed bit flip results in complete mission failure, such as an Apollo mission crash. I'm not really sure what Yeti 2022 is exactly, so take this with heaps of salt, but it seems like this is a "mission failure" event -- it can no lon…
Most likely not. But the tech behind FBA (Federated Byzantine Agreement) distributed ledgers would make an extremely reliable system that can handle malfunction of hardware and large outages of nodes. And since this is a write-only log and only some entities can write to it, it could be implemented with permission so that the system doesn't have to deal with attacks that public blockchain would face.
The push for crypto without ecc ram is a nonstop horror show. Software under normal circumstances is remarkably resilient to having its memory corrupted. However crypto algorithms are designed so that a single bit flip effectively changes all the bits in a block. If you chain blocks then a single bit flip in one block destroys all the blocks. I've seen companies like msps go out of business because they were doing cr…
Distributed, byzantine fault tolerant state machines solve that. At worst, a single node will go out of sync.
Earlier quoted context omitted.
Safety critical systems are not a good fit for a blockchain-based resolution to the Byzantine General problem. Safety critical systems need extremely low latency to resolve the conflict fast. So blockchain is not going to be an appropriate choice for all critical applications when there are multiple low-latency solutions for BFT at a very low ms latency and IIRC, microseconds for avionics systems.
Not sure what you mean with "safety critical". I made no such assumption. Also since the topic is about a (write-only) log it probably doesn't need such low latency. What it more likely needed is final states so once a entry is made and accepted it must be final and ofc correct. DLTs can do this distributed and self-fixing i.e. a node that tries to add fault data is overruled and can never get a confirmation for a fi…
I thought you were making a more general statement about all critical systems, that's all. And since many critical systems have a safety factor in play, I wanted to distinguish them as not always being a good target for a Blockchain solution to the problems of consensus.
Blockchain is a very interesting solution to the problem of obtaining consensus in the face of imperfect inputs, there are other options so, like anything else, you choose the right tool for the job. My own view is that-- given other established protocols, blockchain is going to be overkill for dealing with some types of fault tolerance. It is a very good fit for applications where you want to minimize relying on the trust of humans. (And other areas too, but right now I'm just speaking of the narrow context of consensus amid inconsistent inputs)
Earlier quoted context omitted.
This is just learned helplessness because Intel were stingy as shit for over a decade and wanted to segregate their product lines. Error correction is literally prevalent in every single part of every PHY layer in a modern stack, it is an absolute must, and the lack of error correction in RAM is, without question, a ridiculous gap that should have never been allowed in the first place in any modern machine, especiall…
> By definition, if a failure occurs, it's because it passed multiple safeguards that were already in place. Having worked on a good bunch of critical system, there aren't multiple safeguards in most hardware. E.g. a multiplication error in a core will not be detected by an external device. Or a bit flip when reading cache, or from a storage device. Very often the only real safeguard is to do the whole computation tw…
The quote I made is actually very specifically is in reference to this paper, in particular point 2, which is mandatory reading for any systems engineer, IMO, though perhaps the word "safeguard" is too strong for the taste of some here. But focusing on definitions of words is besides the point and falls into the same traps this paper mentions: https://how.complexsystems.fail/
Back to the original point: is ECC the single solution to this catastrophe? No, probably not. Systems are constantly changing and failure is impossible to eliminate. Design decisions and a number of other decisions could have mitigated it and caused this failure to not be catastrophic. Another thing might also cause it to topple. But let's not pretend like we don't know what we're dealing with, either, when we've already built these tools and know they work. We've studied ECC plenty! You don't need to carry water for a corporation trying to keep its purse filled to the brim (by cutting costs) to proclaim that failure is inevitable and most things chug on, regardless. We already know that much.