Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

As Digg Struggles, VP Of Engineering Is Shown The Door

techcrunch.com

Post reply on HN
Print view View original

91–100 of 124 posts

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#91

Post by ketralnis »

pavs wrote: post #62
cookiecaper wrote: post #15

Earlier quoted context omitted.

They are planning to transition to Cassandra as the primary database. PostgreSQL is not used in a relational way on reddit -- it is used as a makeshift k-v store. You can take a look at this yourself as reddit is fully open-source: http://github.com/reddit .

According to reddit admins some of the recent downtimes in the last few months were due to cassandra and they were having some bad performance and stability issues. I can look it up if anyone is interested but it will be a bit of a work, since they stated it (several times) in the comment section and perhaps once on a blog post too.

That would be me, and you're misquoting me.

I screwed up our Cassandra deployment, and wrote about how I screwed it up. We were under-provisioned, and the version we were using didn't deal with the case of being overloaded in a graceful way. We're no longer under-provisioned, so I don't know if more recent versions deal with it better.

We've never claimed to have performance issues, I don't know where you're getting that one.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#92

Post by ketralnis »

zzzeek wrote: post #59
cookiecaper wrote: post #15

Earlier quoted context omitted.

They are planning to transition to Cassandra as the primary database. PostgreSQL is not used in a relational way on reddit -- it is used as a makeshift k-v store. You can take a look at this yourself as reddit is fully open-source: http://github.com/reddit .

> They are planning to transition to Cassandra as the primary database. do you have a source for that ? > it is used as a makeshift k-v store That is very true, they should switch. It just seemed like they had no plan to do so.

> > They are planning to transition to Cassandra as the primary database.

> do you have a source for that ?

Yes. Me.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#94

Post by pavs »

ketralnis wrote: post #91
pavs wrote: post #62

Earlier quoted context omitted.

According to reddit admins some of the recent downtimes in the last few months were due to cassandra and they were having some bad performance and stability issues. I can look it up if anyone is interested but it will be a bit of a work, since they stated it (several times) in the comment section and perhaps once on a blog post too.

That would be me, and you're misquoting me. I screwed up our Cassandra deployment, and wrote about how I screwed it up. We were under-provisioned, and the version we were using didn't deal with the case of being overloaded in a graceful way. We're no longer under-provisioned, so I don't know if more recent versions deal with it better. We've never claimed to have performance issues, I don't know where you're getting…

yup faulty memory, I tried to be more clear here: http://news.ycombinator.com/item?id=1670968

No malicious intent, after all, I did ask you for verification on twitter, linking here.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#95

Post by ketralnis »

jbellis wrote: post #67
pavs wrote: post #62

Earlier quoted context omitted.

According to reddit admins some of the recent downtimes in the last few months were due to cassandra and they were having some bad performance and stability issues. I can look it up if anyone is interested but it will be a bit of a work, since they stated it (several times) in the comment section and perhaps once on a blog post too.

The cassandra issues were primarily ops failures, and secondarily an older version of Cassandra making it difficult to recover once it was overwhelmed. (Some of the resulting improvements in Cassandra are documented here: http://www.riptano.com/blog/whats-new-cassandra-065 ) Cassandra looks to be working fine for reddit now: http://blog.reddit.com/2010/08/everything-went-better-than-e...

> Cassandra looks to be working fine for reddit now

Yep

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#96

Post by nl »

c00p3r wrote: post #79

Let me guess - the problem is about the difference between theory and practice. In theory, Java is great and Cassandra is great. In practice - Java under a heavy load is a disaster, because it was never designed for it, and Cassandra is a just a hype and propaganda. Face the reality - it doesn't work in production as it supposed to - as a primary storage engine. People at the Digg aren't amateur idiots, so I think th…

> In practice - Java under a heavy load is a disaster

Google's heavy use of serverside Java would indicate otherwise.

> because it was never designed for it

Yes it was. Java has a lot of problems, but one thing that isn't a problem is heavy load.

> Cassandra is a just a hype and propaganda

Facebook seems to be using it ok.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#97

Post by dillydally »

ojbyrne wrote: post #69
TheCondor wrote: post #57

Earlier quoted context omitted.

So what's the digg story here? I get that some VP suggested a new buzzwordy technology, they gave him enough rope to hang himself and he did and left the company with a broken pile of crap. It could happen, if you have a healthy company you give trust to people. That it got this far doesn't speak well of the rest of management. It doesn't speak really well of the rest of the team either. Shouldn't there have been som…

I'm sure the real story is different. My rough guess at it is * the main marketing tool that digg has left is "Kevin Rose as genius" * Kevin and the people he drinks with pushed for the adoption of new cool technology. * the entire thing was a giant clusterfuck because of Kevin and the people he drinks with. * but once that became obvious there was a need for a scapegoat so that digg could keep its primary marketing…

That reads like sour grapes.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#98

Post by scorpion032 »

squidsoup wrote: post #68
amix wrote: post #31

Like Joel Spolsky said it: "They followed the single worst strategic mistake that any software company can make: They decided to rewrite everything from scratch" - http://www.joelonsoftware.com/articles/fog0000000069.html I think rewriting from scratch is the core of their problem and not really Cassandra. Gradually going over to Cassandra would have been a much better idea.

I've heard this time and time again, but what if your application is a genuine ball of mud? Would you really not advocate a rewrite for an unmaintainable spaghetti classic asp app still in production today?

but what if your application is a genuine ball of mud?

Hello, Flash? (via Gruber)

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#99

Post by amorphid »

Silhouette wrote: post #46
soofaloofa wrote: post #21

Earlier quoted context omitted.

Digg isn't a startup. It started in 2004 and has been mature for about 5 years. They are a business now. And a big one at that.

That depends on how you define your terms. I'm not sure a company that has been around 6 years and has ~100 staff can be called a startup any more. On the other hand, I'm not sure a company that still survives more on the wishful thinking of investors than on the money it brings in after 6 years can be called a business, either, and certainly I wouldn't call it either big or mature. I'm honestly not sure what I would…

You call it a small business.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#100

Post by pilif »

MartinCron wrote: post #78
pilif wrote: post #47

Earlier quoted context omitted.

Considering the completely different backends of v3 and v4, I think, this would have been incredibly hard to implement. At least if changes in one system should end up in the other. In this case you would not just have to write scrips to one-time migrate all needed data from v3 mysql to v4 Cassandra. No. You would have to build a mechanism that doesn't just do it in both directions, it would also have to work at near…

In a way, the one-time migration might be harder than near-realtime bidirectional synchronization. That way, you could move portions of users to the new system and back as needed. A sudden leap from one backend to another is like jumping over the grand canyon on a motorcycle. Personally, I would rather build a bridge.

For the concurrent configuration to work consistently, digg would probably have to port v3 to the new backend.

Now in the current case, the most issues apparently come from the non-working backend as opposed to the changed featureset.

So while they could have run the two versions in parallel, they would not have gained anything. Likely, this was their rationale behind not doing so in the first place.

Post reply on HN