Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fake S3 – Save time, money, and develop offline

blog.getspool.com

Post reply on HN
Print view View original

31–40 of 57 posts

Re: Fake S3 – Save time, money, and develop offline

#31

Post by eli »

RandallBrown wrote: post #18

This has little to do with the contents of the article, but I found it interesting. "For development, each engineer runs her own instance of Fake S3 where she can put gigabytes of images and video to develop and test against, and her setup will work offline because it is all local." Is spool a team of all women engineers? (I'm just curious as to whether or not that's true because it's so rare. I don't want to turn th…

The English language lacks an appropriate word for this situation (singular personal pronoun that could apply to either gender). Many people feel using "he" to refer to both genders is sexist and will instead rephrase the sentence or use "they" as a singular (which is controversial). Using "she" is just another option. See http://en.wikipedia.org/wiki/Singular_they#Gender-neutral_la...

Re: Fake S3 – Save time, money, and develop offline

#32

Post by nirvdrum »

Fluxx wrote: post #28
Negitivefrags wrote: post #25

Earlier quoted context omitted.

Isn't this just a method of implementation for your option 3? I don't really see a substantial difference between mocking the server and mocking the API. The same caveat still applies about needing an integration test with the real S3 in either case.

> Isn't this just a method of implementation for your option 3? I don't really see a substantial difference between mocking the server and mocking the API. The short answer is that there is no difference. Just as you could mock out a call to S3API.get(object_id) and have it returns my_object, you could write a server that responds to the S3 API call for getting object_id. The long answer is that using mocks is a lot…

Well, there's at least one pretty major difference: you have persisted data. That might not be ideal for test, although in production you will already have persisted data so test with an empty store might not make sense either. But for a local development or a staging server, being able to recall the data you've stored across processes can be quite handy.

Re: Fake S3 – Save time, money, and develop offline

#34

Post by j45 »

While the triviality of maintaining a sync between the spec and functionality of a Fake and real S3 will show in time, I think this is a neat idea.

One thought that comes to my mind is if I could get away with building entire apps using this and spin it off to S3 where/if it's needed.

Re: Fake S3 – Save time, money, and develop offline

#35

Post by mr_luc »

ben1040 wrote: post #12

I had to do some work on an S3-backed project while out at sea on a cruise ship a few months ago (let's save the discussion about working on vacation for the 501 developer thread). Thanks to git I was able to spool up my commits and then push when I pulled into port and had cellular access, but I wasn't really able to do everything I wanted with the paperclip-backed models without reliable/cheap network access. An of…

On my last project, we used Dragonfly. Holy cow -- trivial to switch between file-backed and s3-backed storage in the various environment config files.

It was a lifesaver, because the wifi at the place I was couch-surfing was a little spotty.

Re: Fake S3 – Save time, money, and develop offline

#36

Post by piggity »

eli wrote: post #31
RandallBrown wrote: post #18

This has little to do with the contents of the article, but I found it interesting. "For development, each engineer runs her own instance of Fake S3 where she can put gigabytes of images and video to develop and test against, and her setup will work offline because it is all local." Is spool a team of all women engineers? (I'm just curious as to whether or not that's true because it's so rare. I don't want to turn th…

The English language lacks an appropriate word for this situation (singular personal pronoun that could apply to either gender). Many people feel using "he" to refer to both genders is sexist and will instead rephrase the sentence or use "they" as a singular (which is controversial). Using "she" is just another option. See http://en.wikipedia.org/wiki/Singular_they#Gender-neutral_la...

I much prefer gender neutral language to using "she" or "he".

Unless of course we are talking about a specific example like "John foolishly unscrewed the cap on his radiator while the engine was running".

In which case I would generally try and mix the gender across examples.

"Jane stabbed herself in the eye with a pencil when she was rear-ended by John on the way to the hospital"

Re: Fake S3 – Save time, money, and develop offline

#37

Post by StavrosK »

jubos wrote: post #6
justinsb wrote: post #2

I'd recommend installing OpenStack's Swift component (S3 equivalent) and evaluating that as well. You can run it on one node for development purposes, you can scale it up if you want private object storage on your network, and many public clouds are offering it: Rackspace Cloud Servers, HP Cloud, AT&T, Korea Telecom, Internap etc Wikipedia use OpenStack Swift to store their images, and have some good presentations on…

Swift is very powerful piece of technology, but it is also more involved to setup. Curious to try RiakCS as well and see how it compares to Swift for running production level S3 object storage.

Isn't RiakCS an online service, like S3 itself? That's what I understand from the basho page...

Re: Fake S3 – Save time, money, and develop offline

#38

Post by dennyabraham »

EricR23 wrote: post #24

Why not just change the storage strategy to saving files locally while in your test environment? Fog lets you do this easily with its configuration options.

I can only imagine this is specific to scenarios where you have to manipulate s3 objects directly and the fog::storage abstraction used for s3 isn't adequate (though I could not example such a scenario specifically)

Re: Fake S3 – Save time, money, and develop offline

#40

Post by dtwwtd »

japherwocky wrote: post #27

there was a python implementation of something like this in tornado (s3server and s3client), though now I don't see it. Anyone follow that project and know what happened to it?

I think what you're talking about is one of Tornado's demos - not sure how current it is though.

https://github.com/facebook/tornado/tree/master/demos/s3serv...

Post reply on HN