Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What does " 2>&1 " mean?

stackoverflow.com

Post reply on HN
Print view View original

221–230 of 260 posts

Re: What does " 2>&1 " mean?

#221

Post by ontouchstart »

momentoftop wrote: post #219
shevy-java wrote: post #215

Earlier quoted context omitted.

And how do you find that? Google search literally is useless for these days, for Average Joe.

Try "info bash" on your system. It's the same manual. In Emacs, when I hit C-h i I get a menu of all my info manuals and I first read the bash one there.

That is correct. And also simple `man bash`:

REDIRECTION Before a command is executed, its input and output may be redirected using a special notation interpreted by the shell. Redirection may also be used to open and close files for the current shell execution environment. The following redirection operators may precede or appear anywhere within a simple command or may follow a command. Redirections are processed in the order they appear, from left to right.

       In the following descriptions, if the file descriptor number is
       omitted, and the first character of the redirection operator is , the redirection
       refers to the standard output (file descriptor 1).

       The word following the redirection operator in the following
       descriptions, unless otherwise noted, is subjected to brace expansion,
       tilde expansion, parameter expansion, command substitution, arithmetic
       expansion, quote removal, pathname expansion, and word splitting.  If
       it expands to more than one word, bash reports an error.

       Note that the order of redirections is significant.  For example, the
       command

              ls > dirlist 2>&1

       directs both standard output and standard error to the file dirlist,
       while the command

              ls 2>&1 > dirlist

       directs only the standard output to file dirlist, because the standard
       error was duplicated as standard output before the standard output was
       redirected to dirlist.
...

Re: What does " 2>&1 " mean?

#222

Post by hielke »

jez wrote: post #51
wahern wrote: post #4

I find it easier to understand in terms of the Unix syscall API. `2>&1` literally translates as `dup2(1, 2)`, and indeed that's exactly how it works. In the classic unix shells that's all that happens; in more modern shells there may be some additional internal bookkeeping to remember state. Understanding it as dup2 means it's easier to understand how successive redirections work, though you also have to know that re…

Another fun consequence of this is that you can initialize otherwise-unset file descriptors this way: $ cat foo.sh #!/usr/bin/env bash >&1 echo "will print on stdout" >&2 echo "will print on stderr" >&3 echo "will print on fd 3" $ ./foo.sh 3>&1 1>/dev/null 2>/dev/null will print on fd 3 It's a trick you can use if you've got a super chatty script or set of scripts, you want to silence or slurp up all of their output,…

With exec you can open file descriptors of your current process.

  if [[ ! -e /proc/$$/fd/3 ]]; then
      # check if fd 3 already open and if not open, open it to /dev/null
      exec 3>/dev/null
  fi
  >&3 echo "will print on fd 3"
This will fix the error you are describing while keeping the functionality intact.

Now with that exec trick the fun only gets started. Because you can redirect to subshells and subshells inherit their redirection of the parent:

  set -x # when debugging, print all commands ran prefixed with CMD:
  PID=$$
  BASH_XTRACEFD=7
  LOG_FILE=/some/place/to/your/log/or/just/stdout
  exec 3> >(gawk '!/^RUN \+ echo/{ print strftime("[%Y-%m-%d %H:%M:%S]  "), $0; fflush() }' >> $LOG_FILE)
  exec > >(sed -u 's/^/INFO:  /' >&3)
  exec 2> >(sed -u 's/^/ERROR: /' >&3)
  exec 7> >(sed -u 's/^/CMD:   /' >&3)
  exec 8>&1 #normal stdout with >&8
  exec 9>&2 #normal stderr with >&9
And now your bash script will have a nice log with stdout and stderr prefixed with INFO and ERROR and has timestamps with the PID.

Now the disclaimer is that you will not have gaurantees that the order of stdout and stderr will be correct unfortunately, even though we run it unbuffered (-u and fflush).

Re: What does " 2>&1 " mean?

#223

Post by momentoftop »

ontouchstart wrote: post #221
momentoftop wrote: post #219

Earlier quoted context omitted.

Try "info bash" on your system. It's the same manual. In Emacs, when I hit C-h i I get a menu of all my info manuals and I first read the bash one there.

That is correct. And also simple `man bash`: REDIRECTION Before a command is executed, its input and output may be redirected using a special notation interpreted by the shell. Redirection may also be used to open and close files for the current shell execution environment. The following redirection operators may precede or appear anywhere within a simple command or may follow a command. Redirections are processed in…

Ah, thanks, didn't realise they put the whole manual into the manpage. For other tools (e.g. make), the info manual is complete but the manpage is just a summary.

Re: What does " 2>&1 " mean?

#224

Post by layer8 »

GuB-42 wrote: post #212
ontouchstart wrote: post #209

Sometime all you need is to RTMF from the source instead of Nth hand information (N > 1) https://www.gnu.org/software/bash/manual/html_node/Redirecti...

Great if you know where to look, but most people who ask themselves the question don't know they have to look up the bash manual in the "redirection" section. The usual thing (before LLMs) is to Google the question, but for the question to appear in Google, someone has to ask it first, and here we are. Also the Stackoverflow answers give different perspectives, context, etc... rather than just telling you what it doe…

> Great if you know where to look, but most people who ask themselves the question don't know they have to look up the bash manual in the "redirection" section.

Where else would you look but in the manual of your shell? And you don’t have to know in which section to look, you can just search for “2>&1” in the bash man page.

Re: What does " 2>&1 " mean?

#225

Post by Steltek »

phailhaus wrote: post #82
agentdrek wrote: post #68

Earlier quoted context omitted.

It should be a lesson to learn on how simple, logical and reliable tools can last decades.

Bash syntax is anything but simple or logical. Just look at the insane if-statement syntax. Or how the choice of quotes fundamentally changes behavior. Argument parsing, looping, the list goes on.

You could make a list of WTFs about any language.

Bash syntax is the pinnacle of Chesterton's Fence. If you can't articulate why it was done that way, you have no right to remove it. Python would be an absolutely unusable shell language.

Re: What does " 2>&1 " mean?

#226

Post by layer8 »

raincole wrote: post #95

The comments on stackoverflow say the words out of my mouth so I'll just copy & paste here: > but then shouldn't it rather be &2>&1? > & is only interpreted to mean "file descriptor" in the context of redirections. Writing command &2>& is parsed as command & and 2>&1 That's where all the confusion comes from. I believe most people can intuitively understand > is redirection, but the asymmetrical use of & throws them…

I agree that it adds to the confusion, but note that `file1>file2` also wouldn’t work (in the sense of “send the output currently going to file1 to file2”) and isn’t symmetrical in that sense as well. Or take `/dev/stderr>/dev/stdout` as the more direct equivalent.

Re: What does " 2>&1 " mean?

#227

Post by Steltek »

nusl wrote: post #77
amelius wrote: post #11

It's a reminder of how archaic the systems we use are. File descriptors are like handing pointers to the users of your software. At least allow us to use names instead of numbers. And sh/bash's syntax is so weird because the programmer at the time thought it was convenient to do it like that. Nobody ever asked a user.

I quite like how archaic it is. I am turned off by a lot of modern stuff. My shell is nice and predictable. My scripts from 15 years ago still work just fine. No, I don't want it to get all fancy, thanks.

For a while, there was a strong trend of "I want to do everything in one singular language". Your coding is in language XYZ. Your build tools will be configured/written in XYZ. Your UI frontend will be generated from XYZ. Everything will be defined in XYZ.

Shell is from a time when you had a huge selection of languages, each for different purposes, and you picked the right one for the job. For complex applications, you would have multiple languages working together.

People look at Bash and think, "I would never dare do $Task with that language!". And you'd be right, because you're thinking you only have one tool in the toolbox.

Re: What does " 2>&1 " mean?

#228

Post by post-it »

skydhash wrote: post #76
post-it wrote: post #62

Earlier quoted context omitted.

Multiple levels of logging, all of which you want to capture but not all in the same place.

Wasn't the idiomatic way the `-v` flag (repeated for verbosity). And then stderr for errors (maybe warning too).

Yes, but sometimes you want just important non-error logs to go to the console or journal, and then those plus verbose logs to go to a file that gets rotated, and then also stderr on top of that.

Re: What does " 2>&1 " mean?

#229

Post by xantronix »

stabbles wrote: post #148
homebrewer wrote: post #143

Earlier quoted context omitted.

This should work? curl --dump-header /dev/fd/xxx https://google.com or mkfifo headers.out curl --dump-header headers.out https://google.com unless I'm misunderstanding you.

Ah yeah, `/dev/fd/xxx` works :) somehow thought that was Linux only.

(Principal Skinner voice) Ah, it's a Bash expression!

Re: What does " 2>&1 " mean?

#230

Post by stephenr »

nazgul17 wrote: post #173
stephenr wrote: post #74

Earlier quoted context omitted.

POSIX has a manual for shell. You can read 99% of it without needing to know any syscalls. I'm not as familiar with it but Bash has an extensive manual as well, and I doubt syscall knowledge is particularly required there either. If your complaint is "I don't know what this syntax means without reading the manual" I'd like to point you to any contemporary language that has things like arrow functions, or operator ove…

No, the complaint is that "the syntax is not intuitive even knowing the simpler forms of redirection": this one isn't a competition of them, but rather an ad-hoc one. I know about manuals, and I have known this specific syntax for half of my life. Arrow functions etc are mechanisms in the language. A template you can build upon. This one is just one special operator. Learn it and use it, but it will serve no other pu…

> the syntax is not intuitive even knowing the simpler forms of redirection

The MDN page for arrow functions in JS has, I shit you not, 7 variations on the syntax. And your complaint is these are not intuitively similar enough?

call > output

call 2>&1

call > output 2> error

call 1> output 2> error

Give me a fucking break.

Post reply on HN