Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unix Commands I Wish I’d Discovered Years Earlier

spin.atomicobject.com

Post reply on HN
Print view View original

221–230 of 259 posts

Re: Unix Commands I Wish I’d Discovered Years Earlier

#222

Post by tripa »

gnaritas wrote: post #185
ben0x539 wrote: post #115

bash(?) thing I wish I'd discovered years earlier: diff file1; command2 > file2; diff file1 file2; rm file1 file2

Yup, cool use of named pipes.

Named pipes are filesystem objects you create with mkfifo.

What's in use here is more widely known as “process substitution”.

Re: Unix Commands I Wish I’d Discovered Years Earlier

#223

Post by dagw »

whitlock wrote: post #202
csense wrote: post #171

My favorite is the parallel jobs feature of xargs. For example, say you want to run a script you wrote called process-video.sh to do some processing on all the video files in a directory (extracting audio to MP3, converting format, etc.). You want to use all 8 of your cores. You could write a Makefile and run it with -j9, or you can do this: find . -name "*.flv" | xargs -n 1 -P 9 ./process-video.sh This immediately f…

If you like xargs, but want more flexibility, I'd highly suggest GNU parallel. Such flexibility includes running jobs on multiple computers, running intensive command using all available CPU's (like xargs -P), and creating unique scripts to handle multiple parameters. http://www.gnu.org/software/parallel/man.html

Parallel also lets you transparently run stuff on remote servers, automatically handling stuff like copying files back and forth. When I have some heavy ad-hoc data processing to do my new favorite trick is spinning up 50-100 ec2 spot instances, point GNU parallel at them and just fire and forget.

Re: Unix Commands I Wish I’d Discovered Years Earlier

#224

Post by Toenex »

sebnukem2 wrote: post #43

How can one use a Unix/Linux system without knowing SSH? How to login remotely without it?

Wow, now I feel old. I thought everyone got started with SunOS on a Sparcstation 2? Seriously, don't most people encounter it either through installing a Linux distribution or firing up a terminal on OSX? Or is even my-first-Unix a virtual experience?

Re: Unix Commands I Wish I’d Discovered Years Earlier

#227

Post by mrb »

WestCoastJustin wrote: post #113
Goladus wrote: post #109

Earlier quoted context omitted.

Great links, though I admit when I saw the OP I was expecting content like that and was pleasantly surprised that they really were "Unix commands I wish I'd discovered years earlier", and not the standard array of unix tips+tricks that you can get by without for awhile but as you become more advanced become second-nature. "Man ascii." The number of times that I wound up generating my own ascii table from whatever lan…

re: ascii tables You reminded me of something. There is actually a really cool and simple site called asciiflow [1], which I use all the time to draw diagrams for explaining things in email, etc. It's pretty cool, and was even submitted several times to HN [2, 3]. [1] http://www.asciiflow.com/#Draw [2] https://news.ycombinator.com/item?id=2847177 [3] https://news.ycombinator.com/item?id=3598177

I can't browse www.asciiflow.com, the name does not resolve:

  $ host www.asciiflow.com
  Host www.asciiflow.com not found: 3(NXDOMAIN)
Caused by both authoritative nameservers for asciiflow.com which return the correct answer (a CNAME to ghs.google.com), but with status=NXDOMAIN (wtf?):

  $ dig +short asciiflow.com ns
  ns2.123-reg.co.uk.
  ns.123-reg.co.uk.
  $    ; > DiG 9.8.1-P1 > @ns2.123-reg.co.uk www.asciiflow.com
  ; (1 server found)
  ;; global options: +cmd
  ;; Got answer:
  ;; ->>HEADER

Re: Unix Commands I Wish I’d Discovered Years Earlier

#228

Post by F30 »

Kiro wrote: post #226

When do you need man ascii and xxd? I'm a front-end developer so I have no clue about these low-level things but I'm interested and would like to learn.

There are a lot of usecases for bot of them, mostly debugging I's say. So I'm gonna tell you what I used them for in the past months:

The ASCII table might be useful when you're dealing with encoding problems and want to know e.g. what exactly happened in a broken URL encoding. I used both of them for binary exploitation in the security class I took at university: When utilizing buffer overflows, it's important to get every byte at the right place of your payload. Sometimes, you also need to treat it as a string and that's where ASCII comes in.

These are only two specific usecases. As I said, there are a lot more. Development of binary file parsers could be another.

Re: Unix Commands I Wish I’d Discovered Years Earlier

#229

Post by kbenson »

nawitus wrote: post #81
fsckin wrote: post #28

One of the more useful bits of ssh is not mentioned: remotely running commands. Example: ssh username@host "echo $HOSTNAME && sudo somecommand && cat somecommand.log" There's probably a better way to do this, but in a pinch I can fix a problem on dozens of machines just by altering the host string.

I think most people know about this, but it's also good to know about scp. You can easily copy files using a ssh connection.

Scp doesn't get all file types correct (fifos, etc), while tar does.

Piped tar commands do, but are generally inferior to rsync (at least if you ever need to copy more than once). I believe I've heard (here) that piped tar commands can be useful for the first sync (possibly better compression, no overhead of file checksums), then rsync thereafter.

Re: Unix Commands I Wish I’d Discovered Years Earlier

#230

Post by helloTree »

aikinai wrote: post #220
helloTree wrote: post #190

My biggest improvement was about finding out about .inputrc and configuring the Bash to use VIM-keybindings which is pretty handy if you are used to the editor. Also the following mapping from ESC to pressing jf via imap jf was very nice as I find it much more ergonomic. And to use it in Bash I have set editing-mode vi set keymap vi $if mode=vi set keymap vi-insert "jf": vi-movement-mode $endif set show-all-if-ambigu…

I have my Bash set to vi keybindings, but I had no idea it was possible to remap them. Now I can finally stop hitting escape in Bash too. Thank you so much!

Yeah it is great if you can use similar commands across applications. Btw. there is also a nice history inside VIM (implemented as a separate buffer) that can be accessed using q: so you can scroll through the history with jfkl. E.g. I use that for accessing files: If you are working on a project you will probably edit similar files. If you use one "programming" vim sessions where you always edit files via :tabe you will find all the recent edited files in your history and using q:/ you can search for them pretty fast. So over time you have automatically a working environment adapted to your current project.
Post reply on HN