tl;dr your memcache branch fragments my consulting clients, you should use my memcache branch instead
Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
31–39 of 39 posts
Re: Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
#32That response from manjuraj was kind of evil
Has open source changed? I remember when people didn't fork projects, rebadge them as there own and then promote them over the original.
Now don't get me wrong, I think its amazing that Twitter are opening up these enhancements to the community, but it feels like a kick in the teeth to the memcached folks to slap a twitter badge on it, why isn't this a collaboration that benefits the whole community? you know, like open source used to work.
I know at 34 I'm a dinosaur in this industry but I do try to keep up with the new way of doing things... This just feels wrong to me.
Re: Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
#33Earlier quoted context omitted.
Admittedly I am not well versed in Rails at present. However, the question in regard to twitter isn't so much whether Rails can scale but whether it could scale when twitter needed it too, which was a fair number of years ago. Did Mongrel or Unicorn even exist back in, say, 2007?
Mongrel did.
Was it mature enough then to have been used in production at twitter's scale?
Re: Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
#34Earlier quoted context omitted.
Mongrel did.
Was it mature enough then to have been used in production at twitter's scale?
I had that thought as well. And honestly I'm not entirely sure which version they were running. By now the community has learned a bunch of lessons and we take some of those for granted.
Still, the guys over at Twitter could have said "We couldn't scale this technology given our requirements and current knowledge so we went with what we are comfortable with, and thats Java." Instead, they blamed Rails. And now any Rails hater brings up Twitter in a flame war. Even though it was some 4-5 years ago.
Re: Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
#35That response from manjuraj was kind of evil
Has open source changed? I remember when people didn't fork projects, rebadge them as there own and then promote them over the original. Now don't get me wrong, I think its amazing that Twitter are opening up these enhancements to the community, but it feels like a kick in the teeth to the memcached folks to slap a twitter badge on it, why isn't this a collaboration that benefits the whole community? you know, like o…
I think open source has changed a bit with the rise of the GitHub era. IMHO, I think @mikeal did a good post regarding this change "Apache considered harmful" http://www.mikealrogers.com/posts/apache-considered-harmful....
Even in the old days, it's easier to fork than work with upstream. I think these days it's just easier share those forks with services like GitHub. It should help spread ideas and improved solutions IMHO so downstream consumers actually benefit.
In Twitter's case, they are planning to do what works for them at the moment: "While we initially focused on the challenging goal of making Memcached work extremely well within the Twitter infrastructure, we look forward to sharing our code and ideas with the Memcached community in the long term."
Re: Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
#36Earlier quoted context omitted.
Are you talking about 15B vs. the visits chart I linked? If so, the 15B number comes from API calls, which do not have to happen through the website (think of all the Twitter clients).
Requests are requests.15B is a gigantic amount.
Right I agree 100%. I just couldn't tell if you were trying to reconcile the 15B with the 45M number from Compete.
Re: Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
#37Earlier quoted context omitted.
Mongrel did.
Was it mature enough then to have been used in production at twitter's scale?
Yes. I was on a call with a couple of the twitter devs around this time. They were running on 30 odd instances at joyent. So while they were decent size, they were nothing like they are now.
Mongrel was always pretty solid actually. It was originally developed for verisign as I recall, and even in its early versions stuck much more closely to the http specs than other webservers.
It caught a lot of flack because of application code blasting out the ruby heap by touching too many objects. Mongrel isn't really to blame for that. With early versions of ActiveRecord it was easy to materialize large result sets without realizing it. A lot of people felt the pain of using associations everywhere without thinking about what would happen when the joins would go north of 10k objects. Not really mongrel's fault.
Re: Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
#38I enjoyed the jab at twitter's "we are the only website on the planet to have scaling issues" holier than thou attitude. Rails is still trying to get over the character assassination by twitter when they failed to scale it. I know first hand that rails can scale very well. Do bad carpenters blame their tools?
Admittedly I am not well versed in Rails at present. However, the question in regard to twitter isn't so much whether Rails can scale but whether it could scale when twitter needed it too, which was a fair number of years ago. Did Mongrel or Unicorn even exist back in, say, 2007?
I believe the main earlier problem was that M:N broadcasting in mysql didn't scale well, but I could be mistaken.
Re: Twitter says they fixed the Memcache calcification problem. Dormando disagrees.
#39Earlier quoted context omitted.
Was it mature enough then to have been used in production at twitter's scale?
I had that thought as well. And honestly I'm not entirely sure which version they were running. By now the community has learned a bunch of lessons and we take some of those for granted. Still, the guys over at Twitter could have said "We couldn't scale this technology given our requirements and current knowledge so we went with what we are comfortable with, and thats Java." Instead, they blamed Rails. And now any Ra…
Twitter is still the largest Rails site on the web as far as I know, so anyone who says that Twitter hates Rails is either being taken out of context or confused.