Live data from Hacker News

Hashpipe – Pipe iff the hash matches

jbenet.github.io

31–40 of 90 posts

Re: Hashpipe – Pipe iff the hash matches

#31
post #25

Deployment one-liners with hashpipe will only work if hashpipe is installed, which would be equally difficult for users to install properly than the software itself. Then you'd need something like this: > Simply install using `curl hashpi.pe | bash`

Hah. Nice one :)

Re: Hashpipe – Pipe iff the hash matches

#33

Keep in mind whatever script you run through this could download additional unverified code.

Not only could it, downloading additional code is often the entire point.

Additionally, it's common for the installer to include things like version numbers, which means the hash will change with each release.

Meteor suffers from the "|sh" install pattern. Creating a Docker packaging of it that felt safe required a lot of extra work as a result:

1) Transforming the installer file into a canonical form free of version numbers. This verifies if the assumptions made about the installer are still valid. It also enables a "latest" tag which installs whatever MDG has currently published.

2) The installer is patched so it checksums the tarball it downloads.

(To be clear, I'm aware of areas both upstream and downstream in the process where unverified code could sneak in easily. But at least I can feel good about the part I'm responsible for.)

Re: Hashpipe – Pipe iff the hash matches

#34
post #29

Since the main use case for this utility is verifying network shell scripts, it would be interesting to see a query param convention, so we could use a tool such as: > hashcurl http://load.this/script?hash= QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8 | sh

When the just-over-the-next-hilltop promised-land nirvana of content-centric networking arrives, the hash will be enough to locate & download the content – so you shouldn't even need an URL: $ hashcurl QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8 | sh Maybe it's even a special filesystem path, that contains (but does not list) everything-that's-nameable-and-findable: $ sh /everything/QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6…

I'm not sure if it's really great or a really terrible, but I love the "/everything" idea.

Re: Hashpipe – Pipe iff the hash matches

#35
post #29

Since the main use case for this utility is verifying network shell scripts, it would be interesting to see a query param convention, so we could use a tool such as: > hashcurl http://load.this/script?hash= QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8 | sh

When the just-over-the-next-hilltop promised-land nirvana of content-centric networking arrives, the hash will be enough to locate & download the content – so you shouldn't even need an URL: $ hashcurl QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8 | sh Maybe it's even a special filesystem path, that contains (but does not list) everything-that's-nameable-and-findable: $ sh /everything/QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6…

It's not save a few characters. It's to allow for flexibility of encoding. "sha1" all but requires ASCII. Sometimes you're limited to hex, base64, etc.

Re: Hashpipe – Pipe iff the hash matches

#37
post #29

Since the main use case for this utility is verifying network shell scripts, it would be interesting to see a query param convention, so we could use a tool such as: > hashcurl http://load.this/script?hash= QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8 | sh

When the just-over-the-next-hilltop promised-land nirvana of content-centric networking arrives, the hash will be enough to locate & download the content – so you shouldn't even need an URL: $ hashcurl QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8 | sh Maybe it's even a special filesystem path, that contains (but does not list) everything-that's-nameable-and-findable: $ sh /everything/QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6…

this already works. install ipfs: http://ipfs.io/docs/install then:

    ipfs init
    ipfs daemon &
    sleep 20  # sorry this will go away
    ipfs mount
    sh /ipfs/QmTpnQL97XEHmyt54mgEwf5BN8gJWvw4sGgwQzjqtBwLX6
you can see it on the web at: http://gateway.ipfs.io/ipfs/QmTpnQL97XEHmyt54mgEwf5BN8gJWvw4...

Re: Hashpipe – Pipe iff the hash matches

#38
post #5

Be warned that this reads everything until EOF into memory, doing something like: hashpipe QmUJPTFZnR2CPGAzmfdYPghgrFtYFB6pf1BqMvqfiPDam8 will produce unexpected behaviour.

yeah, hashes are decided by every single bit including the last one, and not a single bit should be output until the hash matches. It currently buffers everything in memory, but might do this: https://github.com/jbenet/hashpipe/issues/1 (some settings dont have disk though). hashpipe is intended for most executable use cases (usually under <50MB)

Another fun option would be to use a tree hash. The distributor of the content-to-be-verified would then envelope it in a format (to-be-defined) that includes proofs-up-to-root every N bytes. Then the verifier could stream, and know that everything it emits fits into the target hash, needing only N bytes of working space.

Caveats:

The source doing-the-enveloping will need two passes (and enough working space for the remainder-tree).

An attacker could still choose the moment-when-content-goes-bad; in the envisioned use of immediately-executing the verifier output, this might leave things in a problematic/resource-consumptive state. (Scripts could be hardened against such partial-execution failures.)

Re: Hashpipe – Pipe iff the hash matches

#39
post #4

Earlier quoted context omitted.

yeah, i was going to call it `hashcat` -- maybe i should do that. or `pipehash`.

already have a hashcat purring on my computer... https://hashcat.net/oclhashcat/

all the good names are taken. i guess it's time we give up on not clashing.

Re: Hashpipe – Pipe iff the hash matches

#40
post #36

How does it protect from man-in-the-middle attacks? The man in the middle can simply replace the hash. If there's an additional communication channel to provide the hash you could simply provide the whole command with it.

not everything is hosted in the same place. most binaries use CDNs different from the websites. e.g. github + s3.

(see my other comments on this page-- hashpipe is only meant to raise the bar a little)

Post reply on HN