Unix tricks • Hacker News Board

Re: Unix tricks

#61

Post by kbenson » Thu, Mar 07, 2013, 3:53 PM UTC

appplemac wrote: post #13

tar czf - . | ssh destination "tar xz" To pipe all the contents of your current directory (including dotfiles) to the destination machine. Greetings from LSI-UPC!

I used to use this, because it's awesome that it got all the file types correct (sockets, for example).

Then I learned to love rsync -av. The benefits are many-fold.

Re: Unix tricks

#62

Post by mikegioia » Thu, Mar 07, 2013, 3:55 PM UTC

'ssh -R 12345:localhost:22 server.com "sleep 1000; exit"' forwards server.com's port 12345 to your local ssh port, even if you machine is not externally visible on the net.

This one blew my mind

Re: Unix tricks

#63

Post by kbenson » Thu, Mar 07, 2013, 3:55 PM UTC

etrain wrote: post #55

His last trick - compressed fie transfer without intermediate state: 'tar cz folder/ | ssh server "tar xz"' Can be pulled off with two flags to scp - and you get to see progress as a benefit! scp -Cr folder server:dest/

rsync -av folder $USER@$SERVER:/destination/path.

scp won't copy certain file types correctly. Rsync really is better here, and not just because of that.

Re: Unix tricks

#64

Post by niggler » Thu, Mar 07, 2013, 3:55 PM UTC

growt wrote: post #54

If I may add a trick: ctrl-z - stops a program bg - sends the stopped program to the background fg - gets the program back to the foreground (interactive mode) very useful in editor sessions or when you want to get rid of the endless download/scp that is blocking your terminal

Note that each job gets an identifier (which you can see by running `jobs`). Other commands like `kill` can work with the id number by using %[id]. For example:

    $ some_command
    ^Z (hit control z)
    $ some_command_2
    ^Z (hit control z)
    $ jobs
    [1]-  Stopped                 some_command
    [2]+  Stopped                 some_command_2
    $ kill %1
    $ jobs
    [1]-  Terminated: 15          some_command
    [2]+  Stopped                 some_command_2

Re: Unix tricks

#65

Post by gcv » Thu, Mar 07, 2013, 3:56 PM UTC

Learn to use your shell's globbing features instead of overusing find. In zsh, you can do 'print -l /*.(c|cc|h|hh)' for example (I'm sure bash has an equivalent).

Re: Unix tricks

#66

Post by cowmix » Thu, Mar 07, 2013, 3:56 PM UTC

etrain wrote: post #55

His last trick - compressed fie transfer without intermediate state: 'tar cz folder/ | ssh server "tar xz"' Can be pulled off with two flags to scp - and you get to see progress as a benefit! scp -Cr folder server:dest/

I don't think your method preserves ownership, file perms, etc.

Re: Unix tricks

#67

Post by [deleted] » Thu, Mar 07, 2013, 3:57 PM UTC

cowmix wrote: post #66

etrain wrote: post #55

His last trick - compressed fie transfer without intermediate state: 'tar cz folder/ | ssh server "tar xz"' Can be pulled off with two flags to scp - and you get to see progress as a benefit! scp -Cr folder server:dest/

I don't think your method preserves ownership, file perms, etc.

[deleted]

Re: Unix tricks

#68

Post by stevvooe » Thu, Mar 07, 2013, 3:59 PM UTC

kbenson wrote: post #63

etrain wrote: post #55

His last trick - compressed fie transfer without intermediate state: 'tar cz folder/ | ssh server "tar xz"' Can be pulled off with two flags to scp - and you get to see progress as a benefit! scp -Cr folder server:dest/

rsync -av folder $USER@$SERVER:/destination/path. scp won't copy certain file types correctly. Rsync really is better here, and not just because of that.

You need a -z to activate compression. But, nonetheless, please always use rsync when copying host to host.

Re: Unix tricks

#69

Post by ominous_prime » Thu, Mar 07, 2013, 4:02 PM UTC

etrain wrote: post #55

His last trick - compressed fie transfer without intermediate state: 'tar cz folder/ | ssh server "tar xz"' Can be pulled off with two flags to scp - and you get to see progress as a benefit! scp -Cr folder server:dest/

Tar can transfer more filetypes and attributes than scp can (even using -p option). `scp -p` only transfers mode, mtime and atime; you lose ownership, extended atributes, symlinks, and hardlinks.

You will also get better compression with tar (or rsync), as it is compressing the files directly, and not just the ssh stream (-C is just passed on to ssh).

I did the tests years ago, but a quick google found someone who tried to test the various combinations: http://www.spikelab.org/transfer-largedata-scp-tarssh-tarnc-...

Re: Unix tricks

#70

Post by unfletch » Thu, Mar 07, 2013, 4:02 PM UTC

    '!!:n' selects the nth argument of the last command, and '!$' the last arg

A lot of people know about "!$" (which is shorthand for !!:$), but that's just the tip of Bash's history expansion. I use these things all the time. One of my favorite keystroke savers is adding :h, the head modifier, to !$. For example:

    $ cp file.txt /some/annoyingly/deep/target/directory/other.txt
    $ cd !$:h
    $ pwd # => /some/annoyingly/deep/target/directory

Once you understand how each component works it's easier to put them together into new (to you) combinations. For example, once you know that !$ is shorthand for !!:$, it's not a huge leap to reason out that you can use !-2:$ to get the last argument to the 2nd-to-last command. Or !ls:$ for the last arg to the most recent `ls` command.

I also prefer to do substitution with the :s modifier rather than ^ as suggested at the link, for consistency's sake:

    $ echo "foo bar"
    foo bar
    $ echo !!:s/bar/baz
    foo baz
    $ echo !?bar?:s/foo/qux
    qux bar

Relevant Bash manual pages:

http://www.gnu.org/software/bash/manual/html_node/Event-Desi...

http://www.gnu.org/software/bash/manual/html_node/Word-Desig...

http://www.gnu.org/software/bash/manual/html_node/Modifiers....