> SMB is better than NFS. More details would be nice.
1) The caveats of NFS can be crippling if you are a rubbish sysadmin. NFS requires a more thorough understanding than what you can get from a tip sheet.
2) Samba is single threaded, performance will suffer serving SMB from a Linux machine. For this reason it would be better to serve SMB from a Windows machine.
3) Using a foreign protocol between homogenous computers when the native protocol will do is non-ideal. NFS is the right thing when sharing filesystems from Linux to Linux.
If I may add a trick: ctrl-z - stops a program bg - sends the stopped program to the background fg - gets the program back to the foreground (interactive mode) very useful in editor sessions or when you want to get rid of the endless download/scp that is blocking your terminal
Another nice job management tool is disown, which lets you log out of your session without killing the job (similar to starting the command with nohup)
Tar can transfer more filetypes and attributes than scp can (even using -p option). `scp -p` only transfers mode, mtime and atime; you lose ownership, extended atributes, symlinks, and hardlinks. You will also get better compression with tar (or rsync), as it is compressing the files directly, and not just the ssh stream (-C is just passed on to ssh). I did the tests years ago, but a quick google found someone who tr…
In particular, scp is mindblowingly slow on lots of small files. I independently rediscovered the tar-pipe trick while sitting there watching scp laboriously copy thousands of 100-bytes so slowly I could count the files as they went by. That should not be possible, even at modem speeds. Fine for moving one file, OK for directories of very large files, not suitable for general usage where you might encounter a signifi…
Absolutely. Connection latency hits you the hardest, since each file is sent serially, and requires 2 (or 3 with -p) round trips in the protocol, and this is on top of an ssh tunnel with it's own overhead. I can't remember what my tests showed, but I have this inkling feeling that tar over ssh was far faster than rsync for an initial load, since there's no round trips required, but you lose some of possible rsync benefits, like resume-ability and checksums.
This is simply not true: $ mkdir '; echo woops' $ find . -type d -exec echo {} ';' . ./; echo woops As you can see 'woops' is never echoed. EDIT: The reason being the shell is never involved in this process, and the shell is what is responsible for splitting commands on semicolons/newlines.
You are assuming the exact versions of the shell and find programs that you use are the only ones that exist. It may not be a problem on your exact system, but it can be a problem elsewhere.
I am assuming a POSIX-compliant implementation of `find`. The shell is not involved.
FWIW, your `find -print0`/`xargs -0` is not POSIX.
If I may add a trick: ctrl-z - stops a program bg - sends the stopped program to the background fg - gets the program back to the foreground (interactive mode) very useful in editor sessions or when you want to get rid of the endless download/scp that is blocking your terminal
Note that each job gets an identifier (which you can see by running `jobs`). Other commands like `kill` can work with the id number by using %[id]. For example: $ some_command ^Z (hit control z) $ some_command_2 ^Z (hit control z) $ jobs [1]- Stopped some_command [2]+ Stopped some_command_2 $ kill %1 $ jobs [1]- Terminated: 15 some_command [2]+ Stopped some_command_2
Way back in my college days, one of the student admins took down the CS department's server by forgetting the % and killing process 1 by mistake.
I freaked out when I found out about the "screen" command several years back. "screen" starts a virtual screen that you can detach from with "ctrl-a d" and you can log out, login from a different machine/session and reattach with "screen -r". it has history so you can run long running commands and reattach 3 days later to continue from where you left off as if you had been logged in the whole time.
Tmux is the new screen. tmux allows for emacs or vim key bindings to run around the buffer and search, splits horizontal and vertical and easier configuration: http://tmux.sourceforge.net/
rsync -av folder $USER@$SERVER:/destination/path. scp won't copy certain file types correctly. Rsync really is better here, and not just because of that.
You need a -z to activate compression. But, nonetheless, please always use rsync when copying host to host.
rsync is slow if the data is not already on the destination. tar over ssh is fast, and tar over socketpipe is even faster but not encrypted.
I'm not aware of any attributes that tar doesn't preserve.
I freaked out when I found out about the "screen" command several years back. "screen" starts a virtual screen that you can detach from with "ctrl-a d" and you can log out, login from a different machine/session and reattach with "screen -r". it has history so you can run long running commands and reattach 3 days later to continue from where you left off as if you had been logged in the whole time.
Yup! You might want to try tmux, though. I used screen for many years and switched over to tmux about 2 years ago, and have been really happy with it. It (tmux) is also maintained more regularly now; screen development seems to have stagnated.
'ssh -R 12345:localhost:22 server.com "sleep 1000; exit"' forwards server.com's port 12345 to your local ssh port, even if you machine is not externally visible on the net. This one blew my mind
Thanks for calling that one out - had never tried that before and works great. Putting into my bag of tricks ...
The sort | uniq method literally needs to sort the file and pipe it to uniq, a far more memory-intensive operation than the single-pass awk check. You can write your own hash function in AWK if you think you may overstep memory, but of course you risk hash collisions. It's a tradeoff. I tried it on a 1U server with 24GB ram a few years ago and found that the sort was thrashing at the 10GB file size while AWK handled…
Sort will actually externally sort blocks into temp files and merge them. Adjusting this block size can help with thrashing. Awk may still be better for uniquification.