Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Decoded: GNU Coreutils

maizure.org

Post reply on HN
Print view View original

11–20 of 56 posts

Re: Decoded: GNU Coreutils

#11

Post by beefhash »

cat199 wrote: post #4

Or, just browse the originals: https://www.tuhs.org/cgi-bin/utree.pl?file=V7/usr/src https://www.tuhs.org/cgi-bin/utree.pl?file=4.3BSD/usr/src and some of their current decedents: https://svnweb.freebsd.org/ http://cvsweb.openbsd.org/cgi-bin/cvsweb/ http://cvsweb.netbsd.org/bsdweb.cgi/src/?only_with_tag=MAIN https://gitweb.dragonflybsd.org/dragonfly.git conveniently all in one source tree, self hosting, and buildable…

> Or, just browse the originals:

I believe that it's worth noting that doubts about the validity of Caldera's license (at least in some jurisdictions) have been raised recently[1].

[1] https://virtuallyfun.com/wordpress/2018/11/26/why-bsd-os-is-...

Re: Decoded: GNU Coreutils

#12

Post by michaelmior »

Lt_Riza_Hawkeye wrote: post #7
nn3 wrote: post #3

that's fairly cool. Wish such sites were available for more open source packages. Always wanted to know the algorithms used in sort. So ls has more source lines than sort? Funny, but a bit depressing too.

ls is kind of complicated, has a billion flags, and has to print pretty so it has to do some ioctls to get the terminal width and such, then column things out nicely so they don't overflow. Glad I'm not implementing ls from scratch.

exa[0] dubs itself a "modern" ls written in Rust. Interesting to compare. Still over 6,000 lines not including comments.

[0] https://github.com/ogham/exa

Re: Decoded: GNU Coreutils

#13

Post by wahern »

woodruffw wrote: post #9
AceJohnny2 wrote: post #2

Huh, I'd never heard of FTS before, the functions used by Coreutils to traverse the filesystem: http://man7.org/linux/man-pages/man3/fts.3.html

FWIW, the POSIX equivalents are `ftw` and `nftw`[1]. POSIX 2008 deprecates `ftw`. [1]: https://pubs.opengroup.org/onlinepubs/9699919799//functions/...

ftw/nftw is a crap interface as it relies on C callbacks and isn't re-entrant.[1] Android/Bionic, Darwin/macOS, DragonflyBSD, FreeBSD, glibc, musl libc, OpenBSD, NetBSD, and even Solaris all support the FTS API.[2]

Of the extant Unix environments only AIX and QNX seems to lack it. HP-UX doesn't seem to support it, either, but I don't count HP-UX as extant. In any event it's trivial to copy an FTS implementation from a BSD.

I'm a strong advocate for adherence to POSIX, but in this case there's significant benefit from using FTS and very little, if any, cost.

[1] FTS uses a C callback for its comparator, which can be a headache but not nearly to the same extent as it is with nftw.

[2] It's originally from BSD Reno. It was adopted by glibc, which in turn has forced commercial vendors like Solaris to support it as they're chasing Linux/glibc API compatibility. I would expect AIX to add it eventually.

Re: Decoded: GNU Coreutils

#14

Post by serhart »

ggm wrote: post #10
serhart wrote: post #8

Earlier quoted context omitted.

I'm pretty curious on how you conclude that GNU is "intentionally obfuscating code" from [1]. I read [1] as good advice to avoid inadvertently getting code into GNU that could be claimed by copyright. Focus on speed instead of memory; simplicity instead of speed. I don't see "make it different for difference sake." I also conclude the opposite from the cat implementations. I really don't see how the BSD cat is a more…

I really don't see how the BSD cat is a more "pure" or straight forward implementation. I find the GNU cat to have way more options Way more options is "cat -v considered harmful" not thought any more?

If you aren't trying to be Unix I would imagine you aren't thinking that. GNU probably doesn't subscribe to that notion.

Re: Decoded: GNU Coreutils

#15

Post by woodruffw »

wahern wrote: post #13
woodruffw wrote: post #9

Earlier quoted context omitted.

FWIW, the POSIX equivalents are `ftw` and `nftw`[1]. POSIX 2008 deprecates `ftw`. [1]: https://pubs.opengroup.org/onlinepubs/9699919799//functions/...

ftw/nftw is a crap interface as it relies on C callbacks and isn't re-entrant.[1] Android/Bionic, Darwin/macOS, DragonflyBSD, FreeBSD, glibc, musl libc, OpenBSD, NetBSD, and even Solaris all support the FTS API.[2] Of the extant Unix environments only AIX and QNX seems to lack it. HP-UX doesn't seem to support it, either, but I don't count HP-UX as extant. In any event it's trivial to copy an FTS implementation from…

Ah, linux-man bites again: the Linux versions of `nftw` and `ftw` claim to be re-entrant, so I assumed that POSIX specified them as such. Looks like I was wrong about that.

Re: Decoded: GNU Coreutils

#16

Post by enriquto »

this is very cool. Yet GNU utils have so many options that the interesting bits are difficult to find. The same display but for openbsd utils would be still more enlightening.

Re: Decoded: GNU Coreutils

#18

Post by rwmj »

wahern wrote: post #13
woodruffw wrote: post #9

Earlier quoted context omitted.

FWIW, the POSIX equivalents are `ftw` and `nftw`[1]. POSIX 2008 deprecates `ftw`. [1]: https://pubs.opengroup.org/onlinepubs/9699919799//functions/...

ftw/nftw is a crap interface as it relies on C callbacks and isn't re-entrant.[1] Android/Bionic, Darwin/macOS, DragonflyBSD, FreeBSD, glibc, musl libc, OpenBSD, NetBSD, and even Solaris all support the FTS API.[2] Of the extant Unix environments only AIX and QNX seems to lack it. HP-UX doesn't seem to support it, either, but I don't count HP-UX as extant. In any event it's trivial to copy an FTS implementation from…

Annoyingly FTS was completely broken on 64 bit platforms in glibc before 2016. This still affects some enterprise systems (RHEL 7 in my case). Coreutils uses gnulib to replace glibc FTS with a working version on these systems.

Re: Decoded: GNU Coreutils

#19

Post by Anthony-G »

woodruffw wrote: post #15
wahern wrote: post #13

Earlier quoted context omitted.

ftw/nftw is a crap interface as it relies on C callbacks and isn't re-entrant.[1] Android/Bionic, Darwin/macOS, DragonflyBSD, FreeBSD, glibc, musl libc, OpenBSD, NetBSD, and even Solaris all support the FTS API.[2] Of the extant Unix environments only AIX and QNX seems to lack it. HP-UX doesn't seem to support it, either, but I don't count HP-UX as extant. In any event it's trivial to copy an FTS implementation from…

Ah, linux-man bites again: the Linux versions of `nftw` and `ftw` claim to be re-entrant, so I assumed that POSIX specified them as such. Looks like I was wrong about that.

Red Hat-based distributions include POSIX man pages out of the box.

    $ whatis ftw
    ftw (3)              - file tree walk
    ftw (3p)             - traverse (walk) a file tree
Run `man 3p ftw` to read the POSIX version.

On Ubuntu systems, the following packages can be installed to provide the POSIX man pages:

    manpages-posix
    manpages-posix-dev

Re: Decoded: GNU Coreutils

#20

Post by OJFord »

Oh, this is great! And it's one piece of something I've been looking for (so hopefully a good comment section to ask for more): a book on how (GNU/)Linux 'works'.

i.e. I'm not interested in a book of commands or cheat sheets, I can use `man` and SO, Arch wiki, etc. for that when needed.

I think part of the problem is that I don't know what I don't know - but I discovered namespaces (`/proc//ns/`) recently and don't know much about it but thought it was interesting, and managed to use it to do what I needed to overcome a problem I was having with `ip netns`. A similar 'aha' was with inodes a while ago.

So, any recommendations for a book on 'how Linux works'? I think it's a gap in my understanding (academically EE/CS - hardware, up to OS but not Linux-specific, theoretical CS; professionally software, 'using' Linux). Best candidate I've found is Brian Ward's 'How Linux Works: what every superuser should know'.

Post reply on HN