Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unix tricks

mmb.pcb.ub.es

Post reply on HN
Print view View original

231–232 of 232 posts

Re: Unix tricks

#231

Post by dspillett »

philsnow wrote: post #181
dspillett wrote: post #158

Earlier quoted context omitted.

Add pv (available in many standard repos these days, from http://www.ivarch.com/programs/pv.shtml if not) into the mix and you get a handy progress bar too.

Unless pv has gotten way more magical since the last time I used it, you also need to tell it how many bytes to expect if you want a progress bar. If it doesn't know how many bytes there will be, it just gives you a "throbber" (which is better than nothing, though).

It depends how you call it.

    cat file | pv | nc ...
and

    gzip
and so forth will result in a throbber as it can't query the pipe for a length.

If you demoggify the first example to:

    pv file | nc ...
you get a progress bar on the sending end without manually specifying a size.

Even without a proper % progress bar, the display can be useful as you can at least see the total sent so far (so if you know the approximate final size you can judge completeness in your head) and the current rate (so you can see it is progressing as expected (so you get some indication of a problem such as an unexpectedly slow network connection, other than it taking too long)).

Re: Unix tricks

#232

Post by kragen »

kragen wrote: post #228
kilburn wrote: post #208

Earlier quoted context omitted.

You are describing a bad implementation of a bloom filter [1]. Anyway, people expect "uniq" to be correct in all cases (i.e., to never filter a unique line). A default implementation where it would possible (even with a minuscule chance) that this doesn't happen would be a recipe for disaster. It may be a cool option though ;) http://en.wikipedia.org/wiki/Bloom_filter

No, this is not a bad implementation of a bloom filter, and the size of the minuscule chance matters; unless SHA-256 has a flaw in it that we don't know about, SHA-256 collisions are far less likely than undetected hardware errors in your computer. The universe contains roughly 2²⁶⁵ protons, 500 protons per distinct SHA-256 value, and has existed for roughly 2⁵⁸ seconds, which means there are roughly 2¹⁹⁸ SHA-256 val…

Another possibly relevant note: there are about 2¹⁴⁹ Planck times per second. None of the above takes into account the Landauer and related quantum-mechanical limits to computation, which may be more stringent.
Post reply on HN