Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The New AWS Command-Line Interface

aws.typepad.com

Post reply on HN
Print view View original

41–50 of 71 posts

Re: The New AWS Command-Line Interface

#41

Post by the_mitsuhiko »

sivers wrote: post #35
gtaylor wrote: post #11

Earlier quoted context omitted.

> The syntax looks quite the same, but Amazon's awscli Python installer has loads of dependencies. I'll have to see if it's worth switching. Why are the dependencies a problem? By combining a handful of smaller, focused modules that each do something well, you can end up with something better than if you were to re-invent the wheel for every need. AWS and the Python dev team are doing a heck of a job on botocore, and…

I just like knowing what's on my system. `pip install awscli` just installed 26 other modules besides awscli. Now I feel a little obliged to go check out those 26, as well, to see what they are. I agree about not re-inventing the wheel. But the amount of stuff installed is definitely a considering factor when choosing between two seemingly identical scripts.

> `pip install awscli` just installed 26 other modules besides awscli. Now I feel a little obliged to go check out those 26, as well, to see what they are.

So? Use a virtualenv and stop worrying. These 26 dependencies will be separately updated and maintained, who knows what warts are sitting in the monolithic perl scripts.

Re: The New AWS Command-Line Interface

#42

Post by the_mitsuhiko »

spullara wrote: post #32
gtaylor wrote: post #11

Earlier quoted context omitted.

> The syntax looks quite the same, but Amazon's awscli Python installer has loads of dependencies. I'll have to see if it's worth switching. Why are the dependencies a problem? By combining a handful of smaller, focused modules that each do something well, you can end up with something better than if you were to re-invent the wheel for every need. AWS and the Python dev team are doing a heck of a job on botocore, and…

The issue is that they can't release them all compiled into one thing. Instead the dependencies have to pollute the rest of the operating system. This is one of those things that Java got right and virtually everything else except npm (local node_modules) got wrong out of the gate. I'd love to have a simple way to package python apps that depend on other python and native libraries without having to install things se…

> The issue is that they can't release them all compiled into one thing. Instead the dependencies have to pollute the rest of the operating system.

Bullshit. Python supports having modules installed into local locations (see virtualenv).

Just `virtualenv ~/.local/lib/aws; ~/.local/lib/aws/bin/pip install awscli; ln -s ~/.local/lib/aws/bin/aws ~/.local/bin/aws` and put `~/.local/bin` into your PATH.

Re: The New AWS Command-Line Interface

#43

Post by cmccabe »

spullara wrote: post #32
gtaylor wrote: post #11

Earlier quoted context omitted.

> The syntax looks quite the same, but Amazon's awscli Python installer has loads of dependencies. I'll have to see if it's worth switching. Why are the dependencies a problem? By combining a handful of smaller, focused modules that each do something well, you can end up with something better than if you were to re-invent the wheel for every need. AWS and the Python dev team are doing a heck of a job on botocore, and…

The issue is that they can't release them all compiled into one thing. Instead the dependencies have to pollute the rest of the operating system. This is one of those things that Java got right and virtually everything else except npm (local node_modules) got wrong out of the gate. I'd love to have a simple way to package python apps that depend on other python and native libraries without having to install things se…

If you don't want the pip dependencies to "pollute the rest of the system," you can just use pip install with the --root option.

Java's CLASSPATH causes enormous pain for end-users. Just read the Hadoop mailing list. The fact that Java doesn't have a sane default for where to put anything or how to manage dependencies is a huge flaw.

Back when I was doing AWS, I just used the C binaries (I forget what they were called) to transfer things to or from S3. I just wanted to avoid installing hundreds of megs of dependencies. We paid money to transfer our AMIs around, after all! Still, a more full-featured tool will no doubt come in handy in some scenarios.

Re: The New AWS Command-Line Interface

#44

Post by hendry »

aws-cli/0.16.0 gripes:

• slow

• crap ton of dependencies, making it a pain to install on Arch https://aur.archlinux.org/packages/aws-cli/ Hint: static build ffs

• broken help. `aws s3 help` instead of `aws help s3` which is more natural to git users

• no glacier support

• no s3 progress for downloads or uploads

• many jarring UX issues https://github.com/aws/aws-cli/issues/305 https://github.com/aws/aws-cli/issues/304

Re: The New AWS Command-Line Interface

#45

Post by imperialWicket »

sparkzilla wrote: post #33

As a semi-technical small business user of AWS I really wonder why so few resources seem to be put to the AWS GUI. For example, why is autoscaling not part of the GUI/AWS console? It's one of the most basic functions and it beggars belief that in this day and age it is done through the command line only. I'd rather not have to pay extra for Ylastic or some other hack when I'm sure Amazon can throw together even the m…

Use Asgard from NetflixOSS (https://github.com/Netflix/asgard). Shameless project plug: for anyone using Hubot, easily get Asgard up and running, then interface with Asgard via Hubot (https://github.com/imperialwicket/hubot-asgard).

* EDIT: that is, use Asgard because it does a great job of managing autoscaling groups.

Re: The New AWS Command-Line Interface

#46

Post by jessaustin »

jevinskie wrote: post #30
vacri wrote: post #24

Interesting - I must have had an older version and not played with it. The version I had did not have 's3 sync', and the help files were a long list of ec2 commands and some boilerplate headings. Reinstalled and now they have shorter, saner commands and real help pages... and they also changed the pager from 'less' (my default pager) to the crappy 'more'... because (apparently) you never want to go back a page in a h…

Amazon doesn't use $PAGER? [0] [0]: https://en.wikibooks.org/wiki/Guide_to_Unix/Environment_Vari...

Well, GP isn't just imagining things. The pager for this is at most the equivalent of crap-ass "more" right out of the box, on systems where all other cli executables have better pagers.

    $ PAGER=/usr/bin/less aws help
...does seem to do the right thing, but why does this program require it when no other program does?

Re: The New AWS Command-Line Interface

#47

Post by saym »

the_mitsuhiko wrote: post #41
sivers wrote: post #35

Earlier quoted context omitted.

I just like knowing what's on my system. `pip install awscli` just installed 26 other modules besides awscli. Now I feel a little obliged to go check out those 26, as well, to see what they are. I agree about not re-inventing the wheel. But the amount of stuff installed is definitely a considering factor when choosing between two seemingly identical scripts.

> `pip install awscli` just installed 26 other modules besides awscli. Now I feel a little obliged to go check out those 26, as well, to see what they are. So? Use a virtualenv and stop worrying. These 26 dependencies will be separately updated and maintained, who knows what warts are sitting in the monolithic perl scripts.

I feel like it's a valid concern for someone to want to know what's on their system. Sure virtualenv is a great solution, and I'm sure the person you're responding to knows about it. But there is a place for skepticism in required dependencies, and perhaps the OP parses Perl better than Python.

Re: The New AWS Command-Line Interface

#48

Post by HeyImAlex »

dencold wrote: post #37

Long-time user of boto[1] here. It has been the go to library to hook your python code into AWS and has a fairly active following on github[2]. One API point that I've found lacking in boto is a "sync" command for S3. Take a source directory and a target bucket and push up the differences ala rsync, that's the dream. Boto gives you a the ability to push/get S3 resources, but I've had to write my own sync logic. So, t…

s3cmd has a sync option that will do this.

If you're looking for something a little more robust, I just released this a couple of days ago-

https://github.com/HeyImAlex/s3tup

It's still in a really early stage but I've been using it to sync and configure my personal site that's hosted on s3 and it's worked well so far.

Re: The New AWS Command-Line Interface

#49

Post by oblio »

Well, a rant:

* Why does each one of them use different parameters for the same stuff? WHATEVER_URL could be REGION (WHATEVER = EC2, ELB, ETC, ...). One uses a config file for ACCESS_KEY_ID, another one wants a environment variable. Plus they use different names for common stuff.

* Why do are the command line arguments named inconsistently across these tools? --whatever, --what-ever

* Why don't they fail on command error? Right now this only happens if there's a configuration problem - if you send a command and there's a problem (like S3 access denied), it still returns 0.

* Why don't they provide synchronous commands? Right now I have to do the polling myself. Super annoying.

Anyway, I've been using the ones included in Amazon Linux - I hope they were the latest version. If the new version fixes this problem, feel free to correct me :)

Re: The New AWS Command-Line Interface

#50

Post by garnaat »

dencold wrote: post #37

Long-time user of boto[1] here. It has been the go to library to hook your python code into AWS and has a fairly active following on github[2]. One API point that I've found lacking in boto is a "sync" command for S3. Take a source directory and a target bucket and push up the differences ala rsync, that's the dream. Boto gives you a the ability to push/get S3 resources, but I've had to write my own sync logic. So, t…

The rsync command uses a combination of file modtimes and file sizes as it's default algorithm. It's very fast and efficient. I agree, though, that like rsync, it would be good to add a --checksum option to the s3 sync command in AWS CLI. Feel free to create an issue on our github site https://github.com/aws/aws-cli so we can track that.
Post reply on HN