A similar tool is available for SDB:
Fake S3 – Save time, money, and develop offline
21–30 of 57 posts
Re: Fake S3 – Save time, money, and develop offline
#22This has little to do with the contents of the article, but I found it interesting. "For development, each engineer runs her own instance of Fake S3 where she can put gigabytes of images and video to develop and test against, and her setup will work offline because it is all local." Is spool a team of all women engineers? (I'm just curious as to whether or not that's true because it's so rare. I don't want to turn th…
Maybe, maybe not, either way I don't think that's why it was phrased that way. I have no idea of the internals of the team, but I've seen the female terms used instead of the male terms more frequently when referencing software engineers than in other fields. It's just how some people write it.
Re: Fake S3 – Save time, money, and develop offline
#23This has little to do with the contents of the article, but I found it interesting. "For development, each engineer runs her own instance of Fake S3 where she can put gigabytes of images and video to develop and test against, and her setup will work offline because it is all local." Is spool a team of all women engineers? (I'm just curious as to whether or not that's true because it's so rare. I don't want to turn th…
Female pronouns are used in many contexts to counteract the too-common use of male pronouns. I assume jubos was doing this because he knows the tech-world is flooded with male pronouns.
Re: Fake S3 – Save time, money, and develop offline
#24Why not just change the storage strategy to saving files locally while in your test environment? Fog lets you do this easily with its configuration options.
Re: Fake S3 – Save time, money, and develop offline
#25In my opinion, having to replicate S3 in development and test isn't the best idea. There are a few problems I see: You have tied yourself to S3's API, you must maintain this "other" S3 by making sure it behaves like the real S3 and your test and development code never actually hits the real API you're using...until staging or production. There are a few better strategies I can see here: 1. For test, use something lik…
Isn't this just a method of implementation for your option 3? I don't really see a substantial difference between mocking the server and mocking the API.
The same caveat still applies about needing an integration test with the real S3 in either case.
Re: Fake S3 – Save time, money, and develop offline
#26I'd recommend installing OpenStack's Swift component (S3 equivalent) and evaluating that as well. You can run it on one node for development purposes, you can scale it up if you want private object storage on your network, and many public clouds are offering it: Rackspace Cloud Servers, HP Cloud, AT&T, Korea Telecom, Internap etc Wikipedia use OpenStack Swift to store their images, and have some good presentations on…
Swift is very powerful piece of technology, but it is also more involved to setup. Curious to try RiakCS as well and see how it compares to Swift for running production level S3 object storage.
http://devstack.org has a script to deploy OpenStack in two lines (git clone the repo, then run the script)
Re: Fake S3 – Save time, money, and develop offline
#27there was a python implementation of something like this in tornado (s3server and s3client), though now I don't see it. Anyone follow that project and know what happened to it?
Re: Fake S3 – Save time, money, and develop offline
#28In my opinion, having to replicate S3 in development and test isn't the best idea. There are a few problems I see: You have tied yourself to S3's API, you must maintain this "other" S3 by making sure it behaves like the real S3 and your test and development code never actually hits the real API you're using...until staging or production. There are a few better strategies I can see here: 1. For test, use something lik…
Isn't this just a method of implementation for your option 3? I don't really see a substantial difference between mocking the server and mocking the API. The same caveat still applies about needing an integration test with the real S3 in either case.
> Isn't this just a method of implementation for your option 3? I don't really see a substantial difference between mocking the server and mocking the API.
The short answer is that there is no difference. Just as you could mock out a call to S3API.get(object_id) and have it returns my_object, you could write a server that responds to the S3 API call for getting object_id.
The long answer is that using mocks is a lot quicker to develop, easier, more straight forward and has faster run time than maintaining a real runnable copy of S3 that behaves the exact same as the real S3. With the fake S3 you're still spending CPU cycles inside your S3 client while it talks HTTP with your fake S3, which slows unit tests down a lot. Plus fake S3 may have slightly different behavior when your S3API library interacts with it, which could lead to really hard to track down bugs later on. Trusting your APIs is what unit testing is all about.
Re: Fake S3 – Save time, money, and develop offline
#29I did this on a smaller scale within our SOA environment. We're told our DEV must connect to everyone else's DEV. The problem is everybody's DEV is unstable, because by nature, everything deployed there is a work in progress. If someone's service goes down, it can prevent me from testing and block my progress.
So early on when I developed a mock web service which could serve mock data based on the service I was calling. As a result, I always knew what data was available, had coherent data (foreign keys across systems were always valid), and whenever I needed to, I can bring a system down and test my own system's rigidity and error messages. It was great. And then we reengineered all the systems and everything changed.
Re: Fake S3 – Save time, money, and develop offline
#30great concept. will use it this weekend.