Live data from Hacker News

Hadoop / MapReduce alternatives for parallel computing?

tompinckney.com

11–12 of 12 posts

Re: Hadoop / MapReduce alternatives for parallel computing?

#12
post #7

Earlier quoted context omitted.

memcached is a poor-man's distributed shared memory system for clusters. We've been layering on top of it to try and fix deficiencies with things like client-side caching, persistence in case memcached drops objects etc. But I was curious if other people had similar problems and how they were solving them.

Sounds like your life would be simpler with Redis if you are using memcached to take state about a computation. Atomic operations on lists and persistence are two good points about it in this context.

Yeah, redis is pretty interesting.

I think we'd have to add some sort of client side caching on top of it so that we're not fetching the same objects over and over. Tend to saturate our network if we don't do that.

The other thing is that I think we'd have to add some sort of object migration so that when redis servers come up or go down we could re-balance where things are stored.

Post reply on HN