We Just Undid Three Months of Dev work. Here's What We Learned. • Hacker News Board

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#21

Post by fizx » Wed, Oct 07, 2009, 10:18 PM UTC

stingraycharles wrote: post #3

I think the real lesson to learn here is to never assume anything. They assumed performance wouldn't be a problem, so they didn't test for it when prototyping, and they didn't make those tests part of their development cycle. It's tough, these things, especially if you're talking about a feature you personally appreciate a lot. The article talks about the performance problems leading focus away from actively marketin…

> ... never assume anything ...

Maybe you meant "Question your assumptions, often." You can't get anywhere constantly verifying things ahead of time. In the extreme case, imagine writing a test suite for printf just to be sure it works as advertised.

The way forward for a startup is to make only the right assumptions. Failing that, make as many assumptions as possible, but correct the failed ones in a reasonably short amount of time. Some minimal cost/benefit analysis on the assumption would also be good.

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#22

Post by camccann » Wed, Oct 07, 2009, 10:56 PM UTC

fizx wrote: post #21

stingraycharles wrote: post #3

I think the real lesson to learn here is to never assume anything. They assumed performance wouldn't be a problem, so they didn't test for it when prototyping, and they didn't make those tests part of their development cycle. It's tough, these things, especially if you're talking about a feature you personally appreciate a lot. The article talks about the performance problems leading focus away from actively marketin…

> ... never assume anything ... Maybe you meant "Question your assumptions, often." You can't get anywhere constantly verifying things ahead of time. In the extreme case, imagine writing a test suite for printf just to be sure it works as advertised. The way forward for a startup is to make only the right assumptions. Failing that, make as many assumptions as possible, but correct the failed ones in a reasonably shor…

Failing that, make as many assumptions as possible, but correct the failed ones in a reasonably short amount of time. Some minimal cost/benefit analysis on the assumption would also be good.

As a corollary to this, try to arrange that any failures will occur as quickly and obviously as possible. For instance: Component X, which is essentially a black box to you, will be a small but critical part of your project. You don't have time to extensively test it, so you make assumptions about its behavior. Ideally, you should start integrating Component X into the full project immediately such that it's heavily used in development and testing environments, so that any violations of your assumptions will show up incidentally to other work.

On at least a couple occasions, I've been bitten by assuming that a Component X (which I thought I understood) would do what I expected and thus leaving the final integration until near the end of the project. This sounds like an obvious, easily avoided mistake, but it's surprisingly easy to make in the heat of the moment.

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#23

Post by mr_luc » Wed, Oct 07, 2009, 11:40 PM UTC

    The database operations on the nested data were 
    just taking too much processing power.

Given that this is Rails, and given that it's certainly SQL involved here, I just have to ask (and I know the answer is probably "yes") -- did you try implementing nested sets?

I ask because my experience has been that nested data is (with the kinds of nested data I've been handed, anyway) not a performance problem. Selects and updates of nested data can be as responsive as any range query on flat data, and when it comes to managing the performance of inserting new nodes, deleting or moving subtrees, there are a lot of options depending on what you want to optimize for (like spreading out the range from 1 to the max integer supported and periodically re-packing; that way, inserting leaves and any kind of deletion is as fast as with flat data).

I'm curious about what your nested data looked like. Sorry to get distracted on a minor point, but I'm intrigued! When I'm developing, I just always feel as though I should only worry when I start seeing data that has to be a graph and can't be represented as a tree, but as long as it actually is hierarchical then I won't have to worry about speed too much; but now I'm wondering if that intuition will bite me.

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#24

Post by NathanKP » Thu, Oct 08, 2009, 12:13 AM UTC

stingraycharles wrote: post #3

I think the real lesson to learn here is to never assume anything. They assumed performance wouldn't be a problem, so they didn't test for it when prototyping, and they didn't make those tests part of their development cycle. It's tough, these things, especially if you're talking about a feature you personally appreciate a lot. The article talks about the performance problems leading focus away from actively marketin…

These are some really good points to keep in mind. I'm working on a fairly serious website system for a client right now and just reading over the article and the comments here makes me wonder if I should perhaps think ahead and make sure that I don't have to end up rewriting things.

At the moment I feel that my code is fairly future proof, but the truth of the matter is that when it comes to performance its hard to tell how things will work out in the long run.

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#25

Post by chriskelley » Thu, Oct 08, 2009, 12:23 AM UTC

cmos wrote: post #14

chriskelley wrote: post #9

Earlier quoted context omitted.

Being vocal about imperfections in the development cycle and suggesting improvements that matter to the bottom line is not dissent. It's important to have open communication about this very topic - it improves the pipeline and keeps margins where they need to be. Be confident and prepared with data, and make yourself an asset to your company, not just a keypusher! :)

When discussing the merit's of this feature be open minded that you may not have all the information they had in deciding to implement it. Perhaps this is heavily requested by the sales department or by other customers further up the pipeline. Maybe the company threatened to take their business elsewhere. It's always less obnoxious to approach potentially illogical situations by giving someone the benefit of doubt. S…

Absolutely. It's certainly something that needs to be a dialogue, not at all "I know something you don't, here is why you are messing up."

There are always circumstances that affect decisions that you may not know of. This is one of the reasons I am always encouraging artists/developers to be knowledgeable of their project at a higher level. The more you know about what's going on in the big picture, the more of an asset you can make yourself. It's also important for sanity! The OP sounds like they are stewing daily about disagreeing with the feature -- but if there is in fact a relevant reason, some of that burnout-causing heartache could have been avoided.

Dialogue and accountability two huge keys to the management/artist relationship.

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#26

Post by toadpipe » Thu, Oct 08, 2009, 12:29 AM UTC

camccann wrote: post #22

fizx wrote: post #21

Earlier quoted context omitted.

> ... never assume anything ... Maybe you meant "Question your assumptions, often." You can't get anywhere constantly verifying things ahead of time. In the extreme case, imagine writing a test suite for printf just to be sure it works as advertised. The way forward for a startup is to make only the right assumptions. Failing that, make as many assumptions as possible, but correct the failed ones in a reasonably shor…

Failing that, make as many assumptions as possible, but correct the failed ones in a reasonably short amount of time. Some minimal cost/benefit analysis on the assumption would also be good. As a corollary to this, try to arrange that any failures will occur as quickly and obviously as possible. For instance: Component X, which is essentially a black box to you, will be a small but critical part of your project. You…

In my opinion, a lot of The Software Problem(tm) can be traced back to this tendency to underestimate the difficulty of using magic black box X. Maybe this is because most programmers never try to make a real magic black box X that will be used by random person Y in random situation Z themselves. If they did, they might have a little more respect for the fact that it's really fucking hard, the interface will inevitably have all sorts of corners, and the result is never going to be as magic as you would like.

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#27

Post by evgen » Thu, Oct 08, 2009, 3:16 AM UTC

zaidf wrote: post #20

evgen wrote: post #17

Earlier quoted context omitted.

The problem with this ideas is that you don't get to decide which hard problems matter, but your users will not really know if the feature matters until you have made a real effort at providing the feature. There is a particularly insidious meme going around (usually from the so-called "lean startup" crowd) that building a lame/simple version first to see if people like the feature is how you learn what your users wa…

your half-assed "iteration" has led them to decide that you don't have the chops to solve the problem While this theory sounds good, it is disproven time and again by initial half-baked versions of sites that then go on to take off. Just check the original launch of YouTube, Digg, facebook. Also, a HUGE idea coming from lean startup way is to invest very little in marketing until you have a product users like. You do…

> While this theory sounds good, it is disproven time and again by initial half-baked versions of sites that then go on to take off. Just check the original launch of YouTube, Digg, facebook.

It is easy to disprove the theory if you get to cherry-pick your examples. Would you like me to list the thousands of other companies that had a couple of poorly implemented features masquerading as a "beta" that were stomped into dust by others who worked a bit harder to do the job better?

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#28

Post by davidmathers » Thu, Oct 08, 2009, 4:22 AM UTC

Short version:

In our case, the move from flat data to nested data was the killer...We came up with a sweet way of storing the nested data and abstracting away most of complexities of dealing all kinds of data...However, the load on our database was far more than we envisioned...MySQL.

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#29

Post by nestlequ1k » Thu, Oct 08, 2009, 5:45 AM UTC

Great, as a paying customer of scout they are telling me their going to now focus on getting new customers instead of servicing the ones they have.

Might make sense from a business standpoint, but probably something you dont want to advertise :-).

Re: We Just Undid Three Months of Dev work. Here's What We Learned.

#30

Post by acl » Thu, Oct 08, 2009, 7:10 AM UTC

nestlequ1k wrote: post #29

Great, as a paying customer of scout they are telling me their going to now focus on getting new customers instead of servicing the ones they have. Might make sense from a business standpoint, but probably something you dont want to advertise :-).

Actually we're also able to spend a lot more time now on things that really matter to our customers -- things that customers request, like more graphing options and better support for cloud instances. The cloud functionality is already available, and graphing is coming up fast. It's a real pleasure to finally have time to address these things, in addition to the sales and partnership efforts.

Also, performance is significantly better since we simplified the architecture (http://blog.scoutapp.com/articles/2009/10/01/simplify-get-an...), which benefits customers old and new!