Live data from Hacker News

Uploading to S3 in 18 lines of Shell

gist.github.com

11–20 of 23 posts

Re: Uploading to S3 in 18 lines of Shell

#11
post #4

aws s3 cp src dest... OK curl is installed by default but if you have to install this script anyway, it's only a `pip install` away...

Even if the script itself is not "practical", it's very informative. It makes it quite simple to translate into another language for the times when you don't want to shell out to another process.

Re: Uploading to S3 in 18 lines of Shell

#14
Here's a couple I made as well for upload/download to S3 before Travis CI had Docker containers with support for caching in OSS projects: https://github.com/tabolario/ci-scripts

These use the V4 signature, which I grabbed from an article somewhere on the subject that I can't seem to find anymore :( The cached-bundle script is inspired and based off the one from devise: https://github.com/plataformatec/devise/blob/master/script/c...

Re: Uploading to S3 in 18 lines of Shell

#17

Earlier quoted context omitted.

Some platforms and environments don't have pip handy, or you might not have root access, or you can't install new software for any number of reasons, or the required libraries are not installed. curl and /bin/sh and openssl are more universal, so it's good to be able to do what you need to do without resorting to installing software.

You don't need root to install Python packages with pip. You can install them just for your user. See https://pip.readthedocs.org/en/latest/user_guide.html#user-i...

... right, which is why I listed all the other reasons you might not be able to install a python package.

Re: Uploading to S3 in 18 lines of Shell

#18
post #7

Earlier quoted context omitted.

Exactly my thinking. Nice to have one thing less to install on a new Jenkins node, for example.

Except this won't work with files bigger than 5gb. It might be worth the time to do it right.

I don't think the author intended this script to solve everyone's problems. It's an interesting exercise and can be useful in some situations. If it's not useful for your needs, then as has been pointed out there are many many other options.

Re: Uploading to S3 in 18 lines of Shell

#19
post #7

Earlier quoted context omitted.

Exactly my thinking. Nice to have one thing less to install on a new Jenkins node, for example.

Except this won't work with files bigger than 5gb. It might be worth the time to do it right.

> It might be worth the time to do it right.

The right way is the most simple way for your use case. For files under 5GB, this is one of the right ways.

Re: Uploading to S3 in 18 lines of Shell

#20
post #12
post #2

Searching GitHub for S3SECRET and S3KEY might get interesting if this script gets used a lot.

github now auto notifies AWS if it sees you publish secrets.. Then you get an email from AWS saying your keys are public and replace them asap

That's actually pretty awesome.
Post reply on HN