Fake S3 – Save time, money, and develop offline • Hacker News Board

Re: Fake S3 – Save time, money, and develop offline

#41

Post by zackattack » Thu, Apr 19, 2012, 3:22 AM UTC

dtwwtd wrote: post #40

japherwocky wrote: post #27

there was a python implementation of something like this in tornado (s3server and s3client), though now I don't see it. Anyone follow that project and know what happened to it?

I think what you're talking about is one of Tornado's demos - not sure how current it is though. https://github.com/facebook/tornado/tree/master/demos/s3serv...

if anyone wants to port the Ruby script to python, put it on kickstarter and ill start u off with $50. i want a link to my webpage, CompassionPit, on the page for the final tool though. and one on the kickstarter page if ur feeling generous =)

p.s. anyone think we need a developer-tools kickstarter? paul, lmk before i give it to my friends at tech stars

Re: Fake S3 – Save time, money, and develop offline

#42

Post by dlgeek » Thu, Apr 19, 2012, 4:26 AM UTC

anadiazhernandz wrote: post #23

RandallBrown wrote: post #18

This has little to do with the contents of the article, but I found it interesting. "For development, each engineer runs her own instance of Fake S3 where she can put gigabytes of images and video to develop and test against, and her setup will work offline because it is all local." Is spool a team of all women engineers? (I'm just curious as to whether or not that's true because it's so rare. I don't want to turn th…

Female pronouns are used in many contexts to counteract the too-common use of male pronouns. I assume jubos was doing this because he knows the tech-world is flooded with male pronouns.

I have no problem with this in general... but for the love of god, please don't do what a coworker of mine did recently. In a threat model document, he had A sending HIS messages to B, using HER public key. I automatically translated the names to Alice and Bob (like anyone else would in a threat model), hit the pronouns, and my brain segfaulted.

Re: Fake S3 – Save time, money, and develop offline

#43

Post by BrandonM » Thu, Apr 19, 2012, 5:21 AM UTC

zackattack wrote: post #41

dtwwtd wrote: post #40

Earlier quoted context omitted.

I think what you're talking about is one of Tornado's demos - not sure how current it is though. https://github.com/facebook/tornado/tree/master/demos/s3serv...

if anyone wants to port the Ruby script to python, put it on kickstarter and ill start u off with $50. i want a link to my webpage, CompassionPit, on the page for the final tool though. and one on the kickstarter page if ur feeling generous =) p.s. anyone think we need a developer-tools kickstarter? paul, lmk before i give it to my friends at tech stars

Why does the script need to be in Python? Isn't Fake S3 just a server that you run and connect to from any codebase? I'm curious because Python is the language I use most, but I could easily see myself writing a Fake S3 in golang or clojure or even C.

Re: Fake S3 – Save time, money, and develop offline

#44

Post by andrewflnr » Thu, Apr 19, 2012, 5:26 AM UTC

mr_luc wrote: post #35

ben1040 wrote: post #12

I had to do some work on an S3-backed project while out at sea on a cruise ship a few months ago (let's save the discussion about working on vacation for the 501 developer thread). Thanks to git I was able to spool up my commits and then push when I pulled into port and had cellular access, but I wasn't really able to do everything I wanted with the paperclip-backed models without reliable/cheap network access. An of…

On my last project, we used Dragonfly. Holy cow -- trivial to switch between file-backed and s3-backed storage in the various environment config files. It was a lifesaver, because the wifi at the place I was couch-surfing was a little spotty.

This dragonfly? https://github.com/markevans/dragonfly

It's kind of hard to google for "dragonfly".

Re: Fake S3 – Save time, money, and develop offline

#45

Post by LauriL » Thu, Apr 19, 2012, 5:28 AM UTC

deutronium wrote: post #5

Could you use Eucalyptus for this?

You could, but setting up Eucalyptus is a lot of work, compared to Fake S3, a Ruby tool.

Re: Fake S3 – Save time, money, and develop offline

#46

Post by deepakprakash » Thu, Apr 19, 2012, 5:39 AM UTC

Talk about the timing!

We currently have a setup that needs S3 access to reliably develop/test the app I'm currently working on and I had just sat down planning to remove this dependency, since I will be on the road the next week or so.

This will save me a bunch of time immediately and probably some money later on. Thanks!

Re: Fake S3 – Save time, money, and develop offline

#47

Post by Drbble » Thu, Apr 19, 2012, 5:46 AM UTC

arms wrote: post #22

RandallBrown wrote: post #18

This has little to do with the contents of the article, but I found it interesting. "For development, each engineer runs her own instance of Fake S3 where she can put gigabytes of images and video to develop and test against, and her setup will work offline because it is all local." Is spool a team of all women engineers? (I'm just curious as to whether or not that's true because it's so rare. I don't want to turn th…

Maybe, maybe not, either way I don't think that's why it was phrased that way. I have no idea of the internals of the team, but I've seen the female terms used instead of the male terms more frequently when referencing software engineers than in other fields. It's just how some people write it.

It's an unintended consequence of the stereotype the all women engineers are imaginary.

Re: Fake S3 – Save time, money, and develop offline

#48

Post by mb22 » Thu, Apr 19, 2012, 8:26 AM UTC

jubos wrote: post #13

Fluxx wrote: post #10

In my opinion, having to replicate S3 in development and test isn't the best idea. There are a few problems I see: You have tied yourself to S3's API, you must maintain this "other" S3 by making sure it behaves like the real S3 and your test and development code never actually hits the real API you're using...until staging or production. There are a few better strategies I can see here: 1. For test, use something lik…

Excellent points. We work on the idea of different stages in the test and development pipeline. At different stages mock objects make sense, and at other stages having something like Fake S3 makes more sense. For testing, the first stage would be unit testing. At that stage it is best to mock out your S3 interactions (with something like VCR or WebMock) and use an OO approach to wrap your persistence, so you could sw…

it's fine to mock, but for serious s3 users you need to emulate the exact behavior or you are setting up your users for failure.

Re: Fake S3 – Save time, money, and develop offline

#49

Post by zackattack » Thu, Apr 19, 2012, 10:48 AM UTC

BrandonM wrote: post #43

zackattack wrote: post #41

Earlier quoted context omitted.

if anyone wants to port the Ruby script to python, put it on kickstarter and ill start u off with $50. i want a link to my webpage, CompassionPit, on the page for the final tool though. and one on the kickstarter page if ur feeling generous =) p.s. anyone think we need a developer-tools kickstarter? paul, lmk before i give it to my friends at tech stars

Why does the script need to be in Python? Isn't Fake S3 just a server that you run and connect to from any codebase? I'm curious because Python is the language I use most, but I could easily see myself writing a Fake S3 in golang or clojure or even C.

i don't know golang or clojure or c and this is a selfish-borne request that i figured would be worth making because of global benefits.

Re: Fake S3 – Save time, money, and develop offline

#50

Post by mryan » Thu, Apr 19, 2012, 11:21 AM UTC

zackattack wrote: post #41

dtwwtd wrote: post #40

Earlier quoted context omitted.

I think what you're talking about is one of Tornado's demos - not sure how current it is though. https://github.com/facebook/tornado/tree/master/demos/s3serv...

if anyone wants to port the Ruby script to python, put it on kickstarter and ill start u off with $50. i want a link to my webpage, CompassionPit, on the page for the final tool though. and one on the kickstarter page if ur feeling generous =) p.s. anyone think we need a developer-tools kickstarter? paul, lmk before i give it to my friends at tech stars

A Kickstarter for dev tools would be a great idea. As well as new projects, it could also be beneficial for large changes to existing projects. Some combination of Kickstarter and bounties for GitHub issues would be interesting.

My only concern would be for the potential drama caused by members of the community who find the idea of paying for open-source development abhorrent.