Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What does " 2>&1 " mean?

stackoverflow.com

Post reply on HN
Print view View original

141–150 of 260 posts

Re: What does " 2>&1 " mean?

#141

Post by skywal_l »

goku12 wrote: post #134
wahern wrote: post #4

I find it easier to understand in terms of the Unix syscall API. `2>&1` literally translates as `dup2(1, 2)`, and indeed that's exactly how it works. In the classic unix shells that's all that happens; in more modern shells there may be some additional internal bookkeeping to remember state. Understanding it as dup2 means it's easier to understand how successive redirections work, though you also have to know that re…

This is probably one of the reasons why many find POSIX shell languages to be unpleasant. There are too many syntactical sugars that abstract too much of the underlying mechanisms away, to the level that we don't get it unless someone explains it. Compare this with Lisps, for example. There may be only one branching construct or a looping construct. Yet, they provide more options than regular programming languages us…

There must be a law of system design about this, because this happens all the time. Every abstraction creates a class of users who are powerful but fragile.

People who build a system or at least know how it works internally want to simplify their life by building abstractions.

As people come later to use the system with the embedded abstractions, they only know the abstractions but have no idea of the underlying implementations. Those abstractions used to make perfect sense for those with prior knowledge but can also carry subtle bias which makes their use error prone for non initiated users.

Re: What does " 2>&1 " mean?

#142

Post by homebrewer »

murphyslaw wrote: post #111
Dylan16807 wrote: post #104

Earlier quoted context omitted.

> At least allow us to use names instead of numbers. You can use /dev/stdin, /dev/stdout, /dev/stderr in most cases, but it's not perfect.

> You can use /dev/stdin, /dev/stdout, /dev/stderr in most cases Never ever write code that assumes this. These dev shorthands are Linux specific, and you'll even need a certain minimum Linux version. I cringe at the amount of shell scripts that assume bash is the system interpreter, and not sh or ksh. Always assume sh, it's the most portable. Linux != Unix.

It's a waste of time unless you're specifically targeting and testing mac, all of the BSDs, various descendants of Solaris, and other flavors of Unix. I wrote enough "portable shell" to run into so many quirks and slight differences in flags, in how different tools handle e.g. SIGPIPE.

Adding a new feature in a straightforward way often makes it work only on 4/7 of the operating systems you're trying to support. You then rewrite it in a slightly different way (because it's shell — there's always 50 ways to do the same thing). This gets you to 5/7 working systems, but breaks one that previously worked. You rewrite it yet another way, fixing the new breakage, but another one breaks. Repeat this over and over again, trying to find an implementation that works everywhere, or start adding workarounds for each system. Spend an hour on a feature that should have taken two minutes.

If it's anything remotely complicated, and you need portability, then use perl/python/go.

Re: What does " 2>&1 " mean?

#143

Post by homebrewer »

stabbles wrote: post #127
nothrabannosir wrote: post #109

Earlier quoted context omitted.

The aws cli has a set of porcelain for s3 access (aws s3) and plumbing commands for lower level access to advanced controls (aws s3api). The plumbing command aws s3api get-object doesn't support stdout natively, so if you need it and want to use it in a pipeline (e.g. pv), you would naively do something like $ aws s3api get-object --bucket foo --key bar /dev/stdout | pv ... Unfortunately, aws s3api already prints the…

Would be nice if `curl` had something to dump headers to a third file descriptor while outputting the response on stdout.

This should work?

  curl --dump-header /dev/fd/xxx https://google.com
or

  mkfifo headers.out
  curl --dump-header headers.out https://google.com
unless I'm misunderstanding you.

Re: What does " 2>&1 " mean?

#144

Post by kristopolous »

amelius wrote: post #11

It's a reminder of how archaic the systems we use are. File descriptors are like handing pointers to the users of your software. At least allow us to use names instead of numbers. And sh/bash's syntax is so weird because the programmer at the time thought it was convenient to do it like that. Nobody ever asked a user.

I've long wanted easy, trivial multichannel i/o with duplication

I want to be able to route x independent input and y independent output trivially from the terminal

Proper i/o routing

It shouldn't be hard, it shouldn't be unsolved, and it shouldn't be esoteric

Re: What does " 2>&1 " mean?

#145

Post by jcotton42 »

raincole wrote: post #95

The comments on stackoverflow say the words out of my mouth so I'll just copy & paste here: > but then shouldn't it rather be &2>&1? > & is only interpreted to mean "file descriptor" in the context of redirections. Writing command &2>& is parsed as command & and 2>&1 That's where all the confusion comes from. I believe most people can intuitively understand > is redirection, but the asymmetrical use of & throws them…

PowerShell actually has 7 streams. Success, Error, Warning, Verbose, Debug, Information, and Progress (though Progress doesn't get a number) https://learn.microsoft.com/en-us/powershell/module/microsof...

Re: What does " 2>&1 " mean?

#146

Post by marxisttemp »

HackerThemAll wrote: post #18
amelius wrote: post #11

It's a reminder of how archaic the systems we use are. File descriptors are like handing pointers to the users of your software. At least allow us to use names instead of numbers. And sh/bash's syntax is so weird because the programmer at the time thought it was convenient to do it like that. Nobody ever asked a user.

> bash's syntax is so weird What should be the syntax according to contemporary IT people? JSON? YAML? Or just LLM prompt?

Tcl

Re: What does " 2>&1 " mean?

#147

Post by figmert »

47282847 wrote: post #59
jez wrote: post #51

Earlier quoted context omitted.

Another fun consequence of this is that you can initialize otherwise-unset file descriptors this way: $ cat foo.sh #!/usr/bin/env bash >&1 echo "will print on stdout" >&2 echo "will print on stderr" >&3 echo "will print on fd 3" $ ./foo.sh 3>&1 1>/dev/null 2>/dev/null will print on fd 3 It's a trick you can use if you've got a super chatty script or set of scripts, you want to silence or slurp up all of their output,…

Interesting. Is this just literally “fun”, or do you see real world use cases?

One of my use-cases previously has been enforcing ultimate or fully trust of a gpg signature.

    tmpfifo="$(mktemp -u -t gpgverifyXXXXXXXXX)"
    gpg --status-fd 3 --verify checksums.txt.sig checksums.txt 3>$tmpfifo
    grep -Eq '^\[GNUPG:] TRUST_(ULTIMATE|FULLY)' $tmpfifo
It was a while ago since I implemented this, but iirc the reason for that was to validate that the key that has signed this is actually trusted, and the signature isn't just cryptographically valid.

You can also redirect specific file descriptors into other commands:

    gpg --status-fd 3 --verify checksums.txt.sig checksums.txt 3>(grep -Eq '^\[GNUPG:] TRUST_(ULTIMATE|FULLY)')

Re: What does " 2>&1 " mean?

#148

Post by stabbles »

homebrewer wrote: post #143
stabbles wrote: post #127

Earlier quoted context omitted.

Would be nice if `curl` had something to dump headers to a third file descriptor while outputting the response on stdout.

This should work? curl --dump-header /dev/fd/xxx https://google.com or mkfifo headers.out curl --dump-header headers.out https://google.com unless I'm misunderstanding you.

Ah yeah, `/dev/fd/xxx` works :) somehow thought that was Linux only.

Re: What does " 2>&1 " mean?

#149

Post by xeyownt »

raincole wrote: post #95

The comments on stackoverflow say the words out of my mouth so I'll just copy & paste here: > but then shouldn't it rather be &2>&1? > & is only interpreted to mean "file descriptor" in the context of redirections. Writing command &2>& is parsed as command & and 2>&1 That's where all the confusion comes from. I believe most people can intuitively understand > is redirection, but the asymmetrical use of & throws them…

I don't get the confusion.

You redirect stdout with ">" and stderr with "2>" (a two-letter operator).

If you want to redirect to stdout / stderr, you use "&1" or "&2" instead of putting a file name.

Re: What does " 2>&1 " mean?

#150

Post by trashb »

jpollock wrote: post #45
ifh-hn wrote: post #43

Earlier quoted context omitted.

Haha, I'm even more confused now. I have no idea what dup is...

There are a couple of ways to figure out. open a terminal (OSX/Linux) and type: man dup open a browser window and search for: man dup Both will bring up the man page for the function call. To get recursive, you can try: man man unix (the unix is important, otherwise it gives you manly men)

you may also consider gnu info

  info dup
Post reply on HN