Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

As Digg Struggles, VP Of Engineering Is Shown The Door

techcrunch.com

Post reply on HN
Print view View original

71–80 of 124 posts

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#71

Post by Yaa101 »

amix wrote: post #31

Like Joel Spolsky said it: "They followed the single worst strategic mistake that any software company can make: They decided to rewrite everything from scratch" - http://www.joelonsoftware.com/articles/fog0000000069.html I think rewriting from scratch is the core of their problem and not really Cassandra. Gradually going over to Cassandra would have been a much better idea.

No, what they done wrong is not to make a proper 2 way conversion tool for their database backup. Starting from scratch is no problem if you got your core data covered, then you can always revert back.

Don't ever develop yourself into a one way street!!

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#72

Post by brown9-2 »

epoxy wrote: post #39
cookiecaper wrote: post #14

Is there any reason that Cassandra is the focus of this article? It is really silly and irresponsible to peg a nascent project like that without any reasoning or sources. I'm sure something changed besides just a Cassandra rollout, and wasn't Digg using it on v3 too? I think Cassandra is pretty well tested. There have been lots of super-large-scale deployments. It just seems lame to blame it on that, but I guess mayb…

Part of the reason that Cassandra may be the focus is that Kevin Rose places much of the stability problems on Cassandra's shoulders. Two or three minutes into the most recent diggnation he talks about Cassandra and describes Cassandra as "very beta-stage software" and says how days before the launch at least some of their focus was on fixing "Cassandra problems" rather than issues with Digg v4. http://techcrunch.com…

If true, it's a lame excuse: blame should be placed on the people that decided to use the beta software to power their very-important-to-their-paycheck website

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#73

Post by dasil003 »

squidsoup wrote: post #68
amix wrote: post #31

Like Joel Spolsky said it: "They followed the single worst strategic mistake that any software company can make: They decided to rewrite everything from scratch" - http://www.joelonsoftware.com/articles/fog0000000069.html I think rewriting from scratch is the core of their problem and not really Cassandra. Gradually going over to Cassandra would have been a much better idea.

I've heard this time and time again, but what if your application is a genuine ball of mud? Would you really not advocate a rewrite for an unmaintainable spaghetti classic asp app still in production today?

I'd say it depends on if your engineering talent is vastly superior today than what it was when it was written. Even then it's so risky. It's probably always better to do it incrementally even if it takes twice as long to do so, because you can maintain working software and fix bugs as you go.

The approach I would take is to get the minimum set of engineers who know the most about each major aspect of the code, and put their heads together on what the ideal architecture would be. But rather than building it from scratch, figure out how to implement just one of those pieces now. That way you can decrease entropy in the codebase piecemeal without chucking out all the code at once, which is no doubt full of forgotten assumptions that no one will remember until it's too late.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#74

Post by dasil003 »

pavs wrote: post #63
Devilboy wrote: post #60

Earlier quoted context omitted.

I can understand that Digg needs more people than Reddit (even though their traffic numbers are about the same) but TEN TIMES more people? That's just not sensible. I'm sure Digg can make a nice profit without further growth if they just got rid of about half their staff.

I wonder if its easier to get funding (and later position yourself to get sold out), if you show that you have quite a few people working for you. Personally I have no idea nor do I see any reason for that many people working for digg, but my understanding is that Digg (or rather Kevin Rose) has always been about having the perception of big without actually being that big.

You know what's even better? Having the perception that you are big but without actually hiring 100 fucking people to do it. I mean if that's what you need to impress investors at a cocktail party, you're a pretty sad entrepreneur.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#75

Post by pavs »

dasil003 wrote: post #74
pavs wrote: post #63

Earlier quoted context omitted.

I wonder if its easier to get funding (and later position yourself to get sold out), if you show that you have quite a few people working for you. Personally I have no idea nor do I see any reason for that many people working for digg, but my understanding is that Digg (or rather Kevin Rose) has always been about having the perception of big without actually being that big.

You know what's even better? Having the perception that you are big but without actually hiring 100 fucking people to do it. I mean if that's what you need to impress investors at a cocktail party, you're a pretty sad entrepreneur.

I agree, I don't think highly of kevin as an entrepreneur.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#76

Post by pavs »

jbellis wrote: post #67
pavs wrote: post #62

Earlier quoted context omitted.

According to reddit admins some of the recent downtimes in the last few months were due to cassandra and they were having some bad performance and stability issues. I can look it up if anyone is interested but it will be a bit of a work, since they stated it (several times) in the comment section and perhaps once on a blog post too.

The cassandra issues were primarily ops failures, and secondarily an older version of Cassandra making it difficult to recover once it was overwhelmed. (Some of the resulting improvements in Cassandra are documented here: http://www.riptano.com/blog/whats-new-cassandra-065 ) Cassandra looks to be working fine for reddit now: http://blog.reddit.com/2010/08/everything-went-better-than-e...

This could very well be, I remember one of the problems they were having is not having enough resources for cache or something similar. But I could be wrong I will ask them on twitter and see if they can comment here on it.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#77

Post by MartinCron »

squidsoup wrote: post #68
amix wrote: post #31

Like Joel Spolsky said it: "They followed the single worst strategic mistake that any software company can make: They decided to rewrite everything from scratch" - http://www.joelonsoftware.com/articles/fog0000000069.html I think rewriting from scratch is the core of their problem and not really Cassandra. Gradually going over to Cassandra would have been a much better idea.

I've heard this time and time again, but what if your application is a genuine ball of mud? Would you really not advocate a rewrite for an unmaintainable spaghetti classic asp app still in production today?

I would advocate an incremental rewrite of parts of the unmaintainable spaghetti classic asp until there's none of the spaghetti left. It's easier to rewrite part of a system than an entire system.

Release to production dozens, if not hundreds of times. Releases are non-events, rollbacks are non-events.

A system-wide ground-up rewrite with a big-bang switchover at the end is a classic clusterfuck recipe. It's a shame that so many people think it's a good idea, even in 2010.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#78

Post by MartinCron »

pilif wrote: post #47
dawie wrote: post #35

Earlier quoted context omitted.

Even better: Have a link that says: "Try the new Beta" at the top like gmail and google docs, with a "Go back to Old Version". Implementing this should not be to hard and it allows them to get some feedback.

Considering the completely different backends of v3 and v4, I think, this would have been incredibly hard to implement. At least if changes in one system should end up in the other. In this case you would not just have to write scrips to one-time migrate all needed data from v3 mysql to v4 Cassandra. No. You would have to build a mechanism that doesn't just do it in both directions, it would also have to work at near…

In a way, the one-time migration might be harder than near-realtime bidirectional synchronization. That way, you could move portions of users to the new system and back as needed. A sudden leap from one backend to another is like jumping over the grand canyon on a motorcycle. Personally, I would rather build a bridge.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#79

Post by c00p3r »

Let me guess - the problem is about the difference between theory and practice.

In theory, Java is great and Cassandra is great. In practice - Java under a heavy load is a disaster, because it was never designed for it, and Cassandra is a just a hype and propaganda.

Face the reality - it doesn't work in production as it supposed to - as a primary storage engine.

People at the Digg aren't amateur idiots, so I think they do everything as it described in docs, but the damn thing just doesn't work.

Re: As Digg Struggles, VP Of Engineering Is Shown The Door

#80

Post by code_duck »

soofaloofa wrote: post #21
aaronblohowiak wrote: post #20

Earlier quoted context omitted.

your post implies digg isn't a startup.

Digg isn't a startup. It started in 2004 and has been mature for about 5 years. They are a business now. And a big one at that.

Companies like Etsy insist they are still startups, too - five years, $55 million dollars in funding, tens of millions in revenue, and 130 employees later. I don't buy it, either. They are an established business, but routinely use this 'startup' label as an excuse.
Post reply on HN