Maybe OSX developers should just stop asking people to pipe random scripts into their shell. This is a nice idea with a clever name, but it's kind of a false sense of security.
Hashpipe – Pipe iff the hash matches
71–80 of 90 posts
Re: Hashpipe – Pipe iff the hash matches
#72Earlier quoted context omitted.
oh the "Qm..." string is the base58 representation. you can have a binary packed version as well. check out more at https://github.com/jbenet/multihash/ the important thing here is standardizing the encoding, i.e. there should be a valid repr in {binary, hex, b32, b58, b64, ... bemoji ...} so that, say, if your input field _only takes hex_ you have a way to enter the hash. this problem surfaces when tools expect repr…
'sha1:' (0x736861313a) and other self-describing labels can be represented in hex (etc) just fine.
So... it's readable in one out of dozens of possible encodings? And realistically, in that encoding (ASCII), the part following the readable prefix is unreadable garbage. In exchange for this convenience, you add a four bytes of unreadable prefix instead of one.
I'm not sure I see the point.
I deal with outputs from crypto functions on a daily basis. Never once have I intentionally rendered it as 8-byte ASCII. It's most often in hex. Mixing and matching encodings (e.g., prefixing a hex string with an ASCII string) for a single blob of data is silly and just causes headaches whenever you need to change encodings, either by having some data double-encoded or by having to specially handle the prefix separately.
Re: Hashpipe – Pipe iff the hash matches
#73Earlier quoted context omitted.
'sha1:' (0x736861313a) and other self-describing labels can be represented in hex (etc) just fine.
So... it's readable in one out of dozens of possible encodings? And realistically, in that encoding (ASCII), the part following the readable prefix is unreadable garbage. In exchange for this convenience, you add a four bytes of unreadable prefix instead of one. I'm not sure I see the point. I deal with outputs from crypto functions on a daily basis. Never once have I intentionally rendered it as 8-byte ASCII. It's m…
My hunch is that the overwhelmingly dominant and important use case is where these identifiers appear in URLs, including URL fragments (and potentially in brand-new protocols). A few other important cases are also where they're visible to people, as in the example (or hypothetical) command-lines that kicked off this thread. In those cases, explicitness-at-a-glance helps, as will having one canonical encoding (such as b32, b64, or b58).
Compared to that, adaptation to other constrained systems is a case-by-case issue. And if those systems are already capable of squeezing in these non-native slightly-longer hash-like-strings, then a few more bytes usually won't hurt, or if they do then whatever deep-in-the-bits coder (like yourself) who's shoehorning things in can handle compactification. The display/exchange format should be as casually readable as possible.
Such an ASCII-name-inspired prefix is readable in all encodings. In some, it's just a magic number (but one that any coder can make educated guesses about); but in the one encoding that's likely most important for mutual comprehension between humans (user-exchanged strings and URLs), it's super-duper-readable.
And yes, the prefix should in general have special handling: as the multihash project README notes in an important "warning", the prefix lacks the same distribution as the other bytes. Treating the whole thing as an opaque-but-still-reliable identifier invites indexing misoptimizations right off the bat, and then other later bugs, if any of the hash functions become deprecated, or a new hash is added with variant semantics.
Re: Hashpipe – Pipe iff the hash matches
#74I don't see how this gives additional security. When you run
curl https://project.com/script.sh | sh
1. That the people running the project are trustworthy
2. That the server hasn't been compromised
3. That the CA system will ensure you're talking to the correct server
(I can think of recent news stories where each of those were violated.)
If, instead, you go to `https://project.com`, read the instructions, and paste in the following command...
curl https://project.com/script.sh | hashpipe | sh
Re: Hashpipe – Pipe iff the hash matches
#75I think this is a topic worth raising, I spend so much time auditing the "magic" in open source projects to try and find out how they work behind the scenes, and whether they are doing anything to protect me. For example: Pip (python package manager) tells you to run a "secure" script off the internet [0], where the only real protection is being behind SSL. But, surely I can just check the code of that script? Oh wai…
>so they stress that you need to verify the image signature against their GPG signing key which is distributed via github.
If someone has the ability to MiTM HTTPS connections, they could just as easily MiTM the victim's connection to github to return a different GPG signing key. Also, the protection requires active checking rather than coming 'for free' like it does by downloading via HTTPS, so 95%+ of the users won't bother and won't have any protection.
I think CoreOS took a big step backwards here. Just offer the download links via HTTPS in addition to the GPG key. That way people that don't bother with GPG have protection.
Re: Hashpipe – Pipe iff the hash matches
#76I think this is a topic worth raising, I spend so much time auditing the "magic" in open source projects to try and find out how they work behind the scenes, and whether they are doing anything to protect me. For example: Pip (python package manager) tells you to run a "secure" script off the internet [0], where the only real protection is being behind SSL. But, surely I can just check the code of that script? Oh wai…
>so they stress that you need to verify the image signature against their GPG signing key which is distributed via github. If someone has the ability to MiTM HTTPS connections, they could just as easily MiTM the victim's connection to github to return a different GPG signing key. Also, the protection requires active checking rather than coming 'for free' like it does by downloading via HTTPS, so 95%+ of the users won…
[deleted]
Re: Hashpipe – Pipe iff the hash matches
#77Cool. The use case that comes to mind is knowing you are going to get the file you think it is if you are looking it up on some local or remote hash indexed storage scheme.
It would be nice if we had (it prob exists already) a standard block based scheme so files larger than the RAM could be handled without writing to disk. Making something like a .torrent for every file and using that as the per block checksum and then checking the hash of the concatenated checksums could do it. Right? I'm sure there is a real name for that but idk it.
Really this seems like something that the FS should be asked to do. ZFS seems to be close, but I haven't found out how to address a block by it's hash yet.
Re: Hashpipe – Pipe iff the hash matches
#78I don't see how this gives additional security. When you run curl https://project.com/script.sh | sh ...you're relying on three things: 1. That the people running the project are trustworthy 2. That the server hasn't been compromised 3. That the CA system will ensure you're talking to the correct server (I can think of recent news stories where each of those were violated.) If, instead, you go to ` https://project.co…
You add an extra layer of security, if you receive the hash over a different channel, such as encrypted messaging.
That means, that even if the CA system is broken, you only execute the intended contents.
Re: Hashpipe – Pipe iff the hash matches
#79I don't see how this gives additional security. When you run curl https://project.com/script.sh | sh ...you're relying on three things: 1. That the people running the project are trustworthy 2. That the server hasn't been compromised 3. That the CA system will ensure you're talking to the correct server (I can think of recent news stories where each of those were violated.) If, instead, you go to ` https://project.co…
You add an extra layer of security, if you receive the hash over a different channel, such as encrypted messaging. That means, that even if the CA system is broken, you only execute the intended contents.
> That means, that even if the CA system is broken, you only execute the intended contents.
No it doesn't - how are you installing hashpipe? ;)
Re: Hashpipe – Pipe iff the hash matches
#80I don't see how this gives additional security. When you run curl https://project.com/script.sh | sh ...you're relying on three things: 1. That the people running the project are trustworthy 2. That the server hasn't been compromised 3. That the CA system will ensure you're talking to the correct server (I can think of recent news stories where each of those were violated.) If, instead, you go to ` https://project.co…
There is another assumption:
4. You download the script successfully.
It is trivially easy for a MitM to interrupt the download of the script between two TLS packets, without any CA or server compromises. When you do:
curl https://project.com/script.sh | sh
With hashpipe, you are at least guaranteed to have the complete script before you run it. I still don't like the practice, but it is better.