Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple's libc shells out to Perl to implement wordexp

github.com

Post reply on HN
Print view View original

21–30 of 49 posts

Re: Apple's libc shells out to Perl to implement wordexp

#21

Post by jes »

_pmf_ wrote: post #7
ryan0x00 wrote: post #4

sometimes you have to get shit done

> sometimes you have to get shit done And Apple have always been masters in selling polished, nicely packages shit as the most advanced technology ever conceived.

With respect, is there evidence to support your claim?

My direct experience is that I have received value far in excess of what I have had to pay for the Apple products and services I have purchased over the years.

For example, I'm typing this reply on my PowerBook, which I use all day, every day in my work. I also have my iPhone with me constantly, and use it heavily.

Finally, the Apple brand was said to be worth about $104.7B vs. $62.8B for Microsoft in 2014. [1]

So, I am not saying you are wrong, only that I think some evidence would be helpful in making your case.

[1] http://bgr.com/2014/03/19/apple-vs-google-vs-microsoft-brand...

Re: Apple's libc shells out to Perl to implement wordexp

#22

Post by aaronbrethorst »

Luyt wrote: post #14
cnvogel wrote: post #5

As it's already written in the manual... http://linux.die.net/man/3/wordexp wordexp, wordfree - perform word expansion like a posix-shell So, the implementer decided to take the short route, and just spawn a shell which, essentially, gets passed the input data to wordexp(), to do the work. But, of course, having something that starts with such a comment... /* XXX this is _not_ designed to be fast (...) wordexp is als…

Scroll down, there are many more gems in the comments! ...and what a spelling errors: "This kludge is needed because /bin/sh seems to set IFS to the defualt even if you have set it; We also can't just ignore it because it is hard/unplesent to code around or even a potential security problem because the test suiete explicitly checks to make sure setting IFS 'works'" ESR writes about sloppy spelling: "Write in clear, g…

"...and what a spelling errors...ESR writes about sloppy spelling..."

:-\

Re: Apple's libc shells out to Perl to implement wordexp

#23

Post by lloeki »

mrpippy wrote: post #17
dxq wrote: post #6

This is source code from 2011. Checking http://opensource.apple.com will show that it's not longer implemented this way in 10.10. Here's the current implementation: http://opensource.apple.com/source/Libc/Libc-1044.1.2/gen/Fr...

The current one is a derivative of NetBSD's implementation (where it shells out to use a sh builtin)

License-wise:

- this piece of code is 2BSD-licensed

- it calls out to /bin/sh

- on OS X, /bin/sh is hardlinked to /bin/bash

- OS X's bash is GPLv2

Forking to shell is the only way to reuse bash's code. More often than not I really wish sh were not bash.

Re: Apple's libc shells out to Perl to implement wordexp

#24

Post by userbinator »

cnvogel wrote: post #5

As it's already written in the manual... http://linux.die.net/man/3/wordexp wordexp, wordfree - perform word expansion like a posix-shell So, the implementer decided to take the short route, and just spawn a shell which, essentially, gets passed the input data to wordexp(), to do the work. But, of course, having something that starts with such a comment... /* XXX this is _not_ designed to be fast (...) wordexp is als…

The standard itself at http://pubs.opengroup.org/onlinepubs/9699919799/functions/wo... says "While wordexp() could be implemented entirely as a library routine, it is expected that most implementations run a shell in a subprocess to do the expansion" and gives guidance on how to implement it that way.

Re: Apple's libc shells out to Perl to implement wordexp

#25

Post by SrslyJosh »

geofft wrote: post #9
cnvogel wrote: post #5

As it's already written in the manual... http://linux.die.net/man/3/wordexp wordexp, wordfree - perform word expansion like a posix-shell So, the implementer decided to take the short route, and just spawn a shell which, essentially, gets passed the input data to wordexp(), to do the work. But, of course, having something that starts with such a comment... /* XXX this is _not_ designed to be fast (...) wordexp is als…

"Like a POSIX shell" includes expanding command substitution. wordexp("`echo hi`") should return {"hi", NULL}. So it isn't totally ridiculous to reach for a shell; tracking multiple subprocesses, and nested command substitution (like "$(echo $(echo $(echo foo) $(echo bar)))") is a pain to reimplement correctly. In an ideal world, we'd have a libsh that exposed all of the steps of what /bin/sh does in a nice fashion,…

> In an ideal world, we'd have a libsh that exposed all of the steps of what /bin/sh does in a nice fashion, this function would call one of the libsh functions, and /bin/sh would be a 10-line while (true) { libsh_this(); libsh_that(); }.

THIS, GODDAMNIT, THIS.

Re: Apple's libc shells out to Perl to implement wordexp

#26

Post by nandemo »

ryanlol wrote: post #20
nandemo wrote: post #13

Earlier quoted context omitted.

Performance is the least of my worries here. I'd be more worried about security, and plain dependency management. I mean, let's say this libc thing is supposed to be installed in quite a lot of systems, are we really sure that perl thing is installed in all of those systems? And what if, by some crazy coincidence, perl happens to depend on libc?

Almost everything depends on perl. Unless you're running a very small embedded setup, you'll have extremely hard time living without perl. Circular dependencies are perfectly normal.

I'm aware that a lot of projects use perl for the build. But on runtime? I doubt it.

Re: Apple's libc shells out to Perl to implement wordexp

#27

Post by comex »

Here is a vulnerability in the current version [1] for your amusement:

    #include 
    #include 
    int main() {
        wordexp_t we;
        char *s = "$(($(sleep 10)))";
        printf("->%d\n", wordexp(s, &we, WRDE_NOCMD));
    }
(Since the manual page strongly recommends not trusting wordexp with untrusted input even with WRDE_NOCMD, and based on a code search the function is rarely used in the first place, I don't think it's really sensitive.)

http://opensource.apple.com/source/Libc/Libc-1044.1.2/gen/Fr...

Re: Apple's libc shells out to Perl to implement wordexp

#28

Post by SrslyJosh »

Intermernet wrote: post #2

Sorry, I don't have a Mac to check, but does the OSX build of Perl depend on libc? Also, that code seems to be from 2008 (or at least that's the latest year in the copyright header) despite the commit being from 2012. Does anyone know if this has been updated? Inserting a NUL byte between each word, and at the end doesn't sound like it requires Perl...

> Inserting a NUL byte between each word, and at the end doesn't sound like it requires Perl...

No, it's more clever than that. The shell will actually execute something like this:

/usr/bin/perl -e 'print join(chr(0), @ARGV), chr(0)' -- your input string goes here

So, what happens is that the shell (in this case, bash) performs argument expansion on your input string, then calls perl -e '...' with the expanded words as its arguments. What perl does is join all of those arguments with a NUL byte and spit them back out for the calling process to read, which makes figuring out where each expanded "word" begins and ends really simple.

Could you do this in shell? Probably. Would a correct implementation be as short and easy to understand? Probably not. Therefore, perl.

Re: Apple's libc shells out to Perl to implement wordexp

#29

Post by comex »

nandemo wrote: post #13
melling wrote: post #3

Well, you are warned: /* XXX this is _not_ designed to be fast */

Performance is the least of my worries here. I'd be more worried about security, and plain dependency management. I mean, let's say this libc thing is supposed to be installed in quite a lot of systems, are we really sure that perl thing is installed in all of those systems? And what if, by some crazy coincidence, perl happens to depend on libc?

Apple libc in particular is supposed to be installed on exactly two base systems: OS X and iOS. The former has perl installed by default, and the latter marks wordexp as unavailable.

Re: Apple's libc shells out to Perl to implement wordexp

#30

Post by merijnv »

nandemo wrote: post #26
ryanlol wrote: post #20

Earlier quoted context omitted.

Almost everything depends on perl. Unless you're running a very small embedded setup, you'll have extremely hard time living without perl. Circular dependencies are perfectly normal.

I'm aware that a lot of projects use perl for the build. But on runtime? I doubt it.

It's Apple's libc. They ship always ship Perl on OSX (and presumably iOS too?), so what do they care whether other platforms have to deal with an extra dependency? Their platform has Perl anyway, so the dependency is essentially free.
Post reply on HN