Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

curl | sh

curlpipesh.tumblr.com

Post reply on HN
Print view View original

81–90 of 115 posts

Re: curl | sh

#81

Post by fat0wl »

jordanpg wrote: post #36

I sense that the reason these kinds of things are growing in popularity is that there is a command-line renaissance occurring among certains kinds of developers. I'm guessing that about half of these pages are used principally by Ruby/Rails crowds, for example. This is a natural outgrowth of Rails itself which expects even the novice dev to do everything from the CLI. I also notice a certain amount of machismo surrou…

when I switched from RoR to Java I started to really understand these issues... with RoR I was using lots of open source code (how can someone be expected to read/comprehend all the code in c++ extension modules as well as a mound of Ruby?). I'm not sure what c++ extensions are actually capable of but it just seems like a scary thought to be able to hook this deeply into the system. Personally I wouldn't want to use a lib that invokes some c++ unless it is from a REALLY reliable source.

So yeah the reason I bring up Java is I think there is strength in the notion of the "App Server" in the java world. It's like a partner for the OS that allows for somewhat of a security "air-lock". Even if using Docker or something to scale RoR / simulate the effect of an app server node, I just fundamentally think an OS has too much power.... a web app shouldn't have any access to it basically, except through known/monitored/regulated protocols and points of access. Basically, data can be handed to an "App server process" which in turn does some low-levelish system tasks, but the app itself can't do them.

Personally I think the app server approach also makes deployments/scaling easier and reduces the need for a sysadmin guru. RoR shops (and languages that follow a similar paradigm) really shouldn't be running any apps that handle sensitive info for example without a Linux guru and a senior Rails dev auditing app modules (or at least keeping up-to-date on existing security audits).

It's weird, those first few months of Java when you realize how limited you are by your EE or whatever type of container -- then it starts to feel natural/healthy. When architected properly, I don't think apps will ever suffer from this lack of freedom.

Re: curl | sh

#82

Post by nlo »

RKearney wrote: post #12
TeeWEE wrote: post #5

Note that dowloading an app (.exe for windows, or an apple app) and running it, is just as bad. Downloading an installer, and running it, is also insecure. So the question is... What is the proper alternative? I think the best alternatives are appstores such as found on ios and android, right? However that doesnt really fit for open source. Is the way to install open source software, to download the source and compil…

I think one of the larger issues with curl | sh is what could happen in the event of a network outage or early termination on the connection. For example, if you're downloading a script that has a line like this rm -rf ~/.tmp/foo/bar But the HTTP connection was lost before the entire file was downloaded and `rm -rf ~` was the end of one packet and `/.tmp/foo/bar` was the contents of the other (lost) packet, you're sc…

This can be mitigated by wrapping your whole script in parens, as in:

  #!/usr/bin/env bash
  (
    # real work
  )

Re: curl | sh

#83

Post by zimbatm »

new299 wrote: post #19
zimbatm wrote: post #13

Earlier quoted context omitted.

Both curl|sh and git checkouts tend to end up in production deploys so it's interesting to compare them. Both introduce a point of failure if the remote host becomes unavailable and you need to deploy new machines. The curl|sh method is vulnerable to change from the maintainer or any attacker who gets access to the storage layer. git is also vulnerable but requires a collision attack on SHA-1.

If you're running it in production then that's a problem. But I'd dare say that relying on an external "apt-get install xxx" is pretty bad practice in production (to deploy new machines).

I agree but most companies I know don't have the infrastructure or best-practices in place to avoid it. Copy-pasting the curl|sh is very low effort and will predictably end up in deploy code (if you have any).

Re: curl | sh

#84

Post by facepalm »

sarciszewski wrote: post #76
facepalm wrote: post #75

Earlier quoted context omitted.

So you are a fan of the solution that you have to pay the OS provider a fee so that they certify your app? Sure, that's a possibility, but hardly conductive to the open web.

Nope. Don't try to put words in my mouth, please. apt-get install php-composer is much better than curl https://getcomposer.org/installer | php

How do you get your app into the repository? Maybe for Linux you don't need to pay, but you still need to gain approval of the maintainers. Might be even more costly than simply paying up (as for Apple or Microsoft).

Re: curl | sh

#85

Post by mikeash »

michaelmior wrote: post #77
mikeash wrote: post #73

Earlier quoted context omitted.

Yes, in some cases curl won't produce any output, like if the web server is down, or the connection fails before anything is returned. And yes, it would also happen if curl buffers some of the response and then dies. I don't really see why that's interesting.

The default buffer size is typically the page size, which is typically 4096 bytes. I would expect a large number of these scripts to be less than 4096 bytes meaning curl would output nothing before producing an error and the partial script would never be evaluated.

That's the default buffer size for pipes, which won't matter here. When curl terminates, whatever's buffered in the pipe will be flushed. The only thing that could prevent downloaded data from being received by the shell would be internal buffering in curl, if it does any.

Re: curl | sh

#86

Post by dllthomas »

coriny wrote: post #79
dllthomas wrote: post #72

Earlier quoted context omitted.

'To avoid an extra command use "tee" to output to file while viewing it (i.e. it's "less" and ">" functionality combined).' That's undesirable. First, because you should be making the decision about whether to run the script after you look at it, and if it's already running while you read it then it's probably too late. Second, because you should not be echoing possibly-malicious characters to the terminal - view wit…

"First, because you should be making the decision about whether to run the script after you look at it" - exactly, which is what my suggestion is. All I'm suggesting with tee is that it allows you to read in terminal what you are writing to file in the same command. The OP was downloading the script to view in less, and then downloading it again to execute. This method at least ensures that what you view is what you…

"All I'm suggesting with tee is that it allows you to read in terminal what you are writing to file in the same command."

Gotcha. That makes more sense.

Regarding the second, I don't know whether there are presently any attacks in the wild for any commonly deployed setups (if anyone else does, I'd love to hear about them) but terminals are incredibly messy beasts once you start to throw nonprintable characters at them. Better not to expose that surface area.

Re: curl | sh

#87

Post by michaelmior »

mikeash wrote: post #85
michaelmior wrote: post #77

Earlier quoted context omitted.

The default buffer size is typically the page size, which is typically 4096 bytes. I would expect a large number of these scripts to be less than 4096 bytes meaning curl would output nothing before producing an error and the partial script would never be evaluated.

That's the default buffer size for pipes, which won't matter here. When curl terminates, whatever's buffered in the pipe will be flushed. The only thing that could prevent downloaded data from being received by the shell would be internal buffering in curl, if it does any.

Good point. curl doesn't do any internal buffering. I was thinking that the pipeline should be aborted if the curl exits with a non-zero status, but of course this is not the case.

Re: curl | sh

#88

Post by rpdillon »

TeeWEE wrote: post #5

Note that dowloading an app (.exe for windows, or an apple app) and running it, is just as bad. Downloading an installer, and running it, is also insecure. So the question is... What is the proper alternative? I think the best alternatives are appstores such as found on ios and android, right? However that doesnt really fit for open source. Is the way to install open source software, to download the source and compil…

"appstores such as found on ios and android" are actually simpler versions of Linux package management, which fits very well with open-source. Packages are retrieved over HTTPS and signed by maintainers/authors. For distros like Lunar or Gentoo, the source is retrieved in a similar fashion and compiled locally.

Re: curl | sh

#89

Post by dllthomas »

edwintorok wrote: post #78
dllthomas wrote: post #70

Earlier quoted context omitted.

Because you might be piping known stuff to them. Receiving a partial input stream is not necessarily reliant on networking.

OK, but the detection of partial vs complete script is not as simple as 'does last line have EOL'? There are various builtins that require an end token, like case/esac, if/fi, etc. Do these work properly when truncated at an arbitrary line?

Those control structures do work properly - the shell reads ahead until it finds the end token, and fails if it's absent.

It is true that there remains the problem of potential truncation exactly at the end of a top-level line, but I contend that "it stops running here" is a much easier thing to reason about (and, strictly, could always happen if hit with a SIGKILL anyway) than "does the meaning of this line change if we cut it off in a weird place".

Re: curl | sh

#90

Post by heavenlyhash »

TeeWEE wrote: post #5

Note that dowloading an app (.exe for windows, or an apple app) and running it, is just as bad. Downloading an installer, and running it, is also insecure. So the question is... What is the proper alternative? I think the best alternatives are appstores such as found on ios and android, right? However that doesnt really fit for open source. Is the way to install open source software, to download the source and compil…

I agree, it is difficult to find a proper alternative.

App stores do have some advantages here, today. It's also true that they don't well fit the model for open source. App stores define some root of trust for signing, but just like the CA model for HTTPS certificates, there are issues with that. App stores also bring a host of issues with user freedom. (Even the free ones can have this effect unintentionally: what percent of your coworkers know how to manage apt pinning? If one person on ubuntu 10.04 has an working, installed package, and wants to hand that exact (known working) version of it to a friend on ubuntu 12.04, can they do it?)

PGP signed binaries are one major step in the right direction. More friendly tools and popularized workflows for this would help a lot of people avoid major moments of vulnerability.

PGP (or any other signing system) also leave an interesting challenge, which is what to do in the case of key compromise. Revocation is hard. And say you want an audit log of all prior releases: what can you do?

We're all used to git having an immutable log of history by hash chaining, at this point. Is it time to start using systems like that to keep an auditable log of software releases, as well?

I've been working on something called "mdm" [1] that attempts to do an auditable, immutable chain of releases like this. It was originally designed to be a dependency manager for (arbitrary, language-agnostic, any binaries) software development needs, but I wonder if something like this is what the software distribution world needs to start evolving towards as well. Imagine if your end users could all verify that they have the same hash... and the same picture of historic releases from the same author, and the same picture of if the releaser's signing key changed, and so on.

I'd love to hear other thoughts on how hash chains and signatures can be made to intersect to tell the broadest story. The challenges are very real here.

---

[1] https://github.com/polydawn/mdm/

Post reply on HN