81–90 of 259 posts
One of the more useful bits of ssh is not mentioned: remotely running commands. Example: ssh username@host "echo $HOSTNAME && sudo somecommand && cat somecommand.log" There's probably a better way to do this, but in a pinch I can fix a problem on dozens of machines just by altering the host string.
Earlier quoted context omitted.
There really isn't any way around this. The first pass over your data is going to be slow while the inode data is examined, but it will be cached for the next pass. If you do this before that data gets removed from the cache it will be much faster. imac:~ $ time find . | wc -l 419191 real 0m37.865s user 0m0.385s sys 0m2.611s imac:~ $ time find . | wc -l 419195 real 0m10.503s user 0m0.330s sys 0m1.430s
This is a nasty hack where it is counting the output from find. What do you do when find cannot handle the sheer number of files, argument too long from the linux hardcoded MAX_ARG_PAGES We need a dedicated utility that counts inodes.
Earlier quoted context omitted.
There really isn't any way around this. The first pass over your data is going to be slow while the inode data is examined, but it will be cached for the next pass. If you do this before that data gets removed from the cache it will be much faster. imac:~ $ time find . | wc -l 419191 real 0m37.865s user 0m0.385s sys 0m2.611s imac:~ $ time find . | wc -l 419195 real 0m10.503s user 0m0.330s sys 0m1.430s
This is a nasty hack where it is counting the output from find. What do you do when find cannot handle the sheer number of files, argument too long from the linux hardcoded MAX_ARG_PAGES We need a dedicated utility that counts inodes.
Also, to count the number of files in a directory, you need to do readdir(), which will get you the names. Then, to see if any of them are directories to be recursed into you need to stat() the names individually. This is all "find" does. How, I'd very much like to know, is this "a nasty hack"?
Earlier quoted context omitted.
There really isn't any way around this. The first pass over your data is going to be slow while the inode data is examined, but it will be cached for the next pass. If you do this before that data gets removed from the cache it will be much faster. imac:~ $ time find . | wc -l 419191 real 0m37.865s user 0m0.385s sys 0m2.611s imac:~ $ time find . | wc -l 419195 real 0m10.503s user 0m0.330s sys 0m1.430s
This is a nasty hack where it is counting the output from find. What do you do when find cannot handle the sheer number of files, argument too long from the linux hardcoded MAX_ARG_PAGES We need a dedicated utility that counts inodes.
I know it's not exactly an unknown command, but I didn't know about "sort" until last week. It's freaking fast and convenient, sorts hadoop reduce results like a champ.
The funny part is that one of the early uses of Google's map/reduce approach was probably to distribute the job of sorting search results among many servers. And here you are sorting Hadoop results with something as small and simple as sort(1).
Earlier quoted context omitted.
There really isn't any way around this. The first pass over your data is going to be slow while the inode data is examined, but it will be cached for the next pass. If you do this before that data gets removed from the cache it will be much faster. imac:~ $ time find . | wc -l 419191 real 0m37.865s user 0m0.385s sys 0m2.611s imac:~ $ time find . | wc -l 419195 real 0m10.503s user 0m0.330s sys 0m1.430s
This is a nasty hack where it is counting the output from find. What do you do when find cannot handle the sheer number of files, argument too long from the linux hardcoded MAX_ARG_PAGES We need a dedicated utility that counts inodes.
The output from find is a list of filenames separated by newlines. The wc command is reading from stdin and counting those newlines[1] -- wc itself has just two arguments in its argv array here ("wc", "-l"). You will not exceed its argument list. Further, it is not at all a hack to count the number of files this way; rather using pipes to compose functionality is the Unix way[2].
But fwiw, if you want a count of the inodes in use in the entire filesystem, you can get that directly from "df -i".
In any case, as an exercise to the reader, I encourage you to grab the source to find and add the -count output option you desire.
[1] Technically a filename can contain newlines, so this would throw off the count. The fix for that would be to use find's -print0 output option and then pipe to something which can count on the nulls. In practice, you're unlikely to have such filenames.
One of the more useful bits of ssh is not mentioned: remotely running commands. Example: ssh username@host "echo $HOSTNAME && sudo somecommand && cat somecommand.log" There's probably a better way to do this, but in a pinch I can fix a problem on dozens of machines just by altering the host string.
I want a command that counts files in a tree like du does sizes but without having to pipe find through wc which is crazy for hundred thousand files. Can't they just directly access inodes for high speed counting?
Earlier quoted context omitted.
The single most useful thing about ssh to me is that it connects stdout/in across the channel (this is behavior inherited from rsh). This allows for e.g.: % (cd /foo; tar cpf - .) | ssh bar \(cd /baz\; tar xpf -\) Or: % cat ~/.ssh/id_rsa.pub | ssh bar tee -a .ssh/authorized_keys
Forgive my ignorance, but I don't know what what it means to connect "across the channel", but it sounds important. Do you know a link that explains the concept? I've failed to Google it.
So basically, you get :
(local) (remote)
stdin -> ssh ssh -> stdout