Live data from Hacker News

The other half of "Artists Ship"

paulgraham.com

11–20 of 132 posts

Re: The other half of "Artists Ship"

#11
Some thoughts:

As companies grow big, they like to emulate larger companies, and hence the people they like to hire are from larger companies since they understand scale -- this comes with a lot of baggage ofcourse. This in turn, most times, brings with it a culture of people who are worried more about not screwing up, covering their own ass, and thinking about getting promoted rather than doing what's right for the company or the business and screwing up as a byproduct of making quick decisions.

Re Joel Spolsky: The cost of the sale is not only the cost of the product, but what an organization would pay for a software/service, and what they would value it at. This is why most companies have a sales force, and don't advertise their prices on their website. As Joel mentions, for a lot of companies, charging less than $50k is a rounding error and not worth their time.

The flip side of people at big companies buying is based not entirely on the performance of the service or the product. As long as the service/product is reliable & ok, and it doesn't screw up in any major ways, they won't get fired for making that buying decision, rather than looking at ways of maximizing/optimizing the performance of the product -- yet another reason committees make buying decisions.

Re SOX: I like what founder's fund + facebook is doing -- letting early employees cash some of their equity out, thereby increasing the time early employees will stay with the company.

I also like Fred Wilson's thoughts on a secondary market for startup stock (similar to what goog does), this would in some sense, I guess let you be a private company, and not deal with the challenges associated with SOX compliance and at the same time not raise a highly dilutive Series D, in addition there is some sort of liquidation event for the investors.

PS: (std. Buchheit comments about Limited Life Experiences + Overgeneralization = Advice apply)

PPS: Say hi to the reddit guys for me ;-)

Re: The other half of "Artists Ship"

#12
I think the costs of safety checks can be more pronounced in software than in other areas of business. It's not just that good hackers are particularly irritated by them -- they're irritated for a reason.

A two-week release-process lag is so much worse than it might sound. If you can ship instantly, you can get feedback and make improvements rapidly, over and over. Sure, without a careful release process you might occasionally break things, but when you do you can fix them in minutes rather than weeks. When you insert release-process lag into that cycle, improvements that might have been made in a rapid series of releases over a day or two can take months. Sometimes those improvements just never get made, because no one is motivated to keep working on them for so long.

In software, "stable" often means "unchanging", but only rarely means "high-quality".

Re: The other half of "Artists Ship"

#13
post #5
post #4

One problem is that the cost of checks is only visible in the aggregate. The marginal cost of each new check seems to be pretty low. I wonder if there's a way to limit the total cost without hard-and-fast silly rules.

Is "one in, one out" a silly rule? e.g. you can only add a new step to a process if a step is removed from another.

Well yes, because that assumes that there is some magical amount of checks that must never be exceeded. Some checks are extremely valuable and should be added based on their merits alone.

Re: The other half of "Artists Ship"

#14
post #10
post #7

Earlier quoted context omitted.

Well, as companies get larger they can afford less and less to make mistakes. If I'm running a start-up company, I can get away with changing things abruptly, because I have a smaller base of users and I have fewer people relying on my running at production-level. Also, chances are I'll be able to look at user feedback effectively. Once you get large, and your company is providing sustenance for all your employees, p…

This thinking has always kind of confused me. Why are customers/users at a big company more important than those at a startup? Just because there's more of them, now you can't make mistakes? If you have the agility to make rapid production changes, you also have the ability to rapidly rollback. So the argument that larger companies require more checks and testing than startups isn't really valid, especially when you…

I think this phenomenon is caused by the culture of fear among most employees. Screwing up badly will get you fired, the greatest fear of all. Most employees have heavy mortgages and children, and require a consistent and predictable income. (This state of things is a problem in itself, but that's another topic).

The big issue is that the level of fear increases as you move up the command structure.

Let's assume one of your startups has crashed and burned. Something really spectacular must have happened if this makes potential customers of your next startup spurn you. Taking risks may have consequences, but they are temporary. But if you are a high-level manager for a large company, spectacular failure will make you unemployable, or in the best case push you several steps down the ladder - steps which took years of boredom and politics to climb, and which you will never get back. Failure is punished disproportionately, success is rarely compensated. Conservatism is the correct choice for each individual actor.

We all know bureaucracy, culture and politics - existing companies won't change in this area.

Re: The other half of "Artists Ship"

#15
post #10
post #7

Earlier quoted context omitted.

Well, as companies get larger they can afford less and less to make mistakes. If I'm running a start-up company, I can get away with changing things abruptly, because I have a smaller base of users and I have fewer people relying on my running at production-level. Also, chances are I'll be able to look at user feedback effectively. Once you get large, and your company is providing sustenance for all your employees, p…

This thinking has always kind of confused me. Why are customers/users at a big company more important than those at a startup? Just because there's more of them, now you can't make mistakes? If you have the agility to make rapid production changes, you also have the ability to rapidly rollback. So the argument that larger companies require more checks and testing than startups isn't really valid, especially when you…

No, it's that most people don't want to use a product that changes often, if at all. They get it for what it is and if it changes, it means disrupting their way of using it. When you're working with a smaller product, you can iterate more and work closely with the people using it. When it gets larger, any change you make will alienate some users, so you have to plan to minimize that alienation.

You can change and roll back features quickly, but that gives an impression of instability among users. If things are constantly changing, they'll seek out something more stable. It's "lowest common denominator" thinking. It results in something that nobody dislikes, and that's the goal of larger groups. Niche companies are able to work much better, but even then there's some slowdown.

Re: The other half of "Artists Ship"

#16
post #8

This past summer I worked as an intern for a big company. On the first day of orientation the head of the legal compliance department said very flatly to us that any individual could do far more harm to the firm than good. This mentality, reenforced by the company's bureaucratic change management system, really did not sit well with me. Fortunately for my summer experience, my "buddy"/summer mentor and I found a loop…

For large companies with a valuable reputation, that's almost guaranteed to be true. Most people won't generate a million dollars worth of value in a given year, but nearly everyone could do that amount of damage to their company's reputation in just a few minutes.

Re: The other half of "Artists Ship"

#17
post #10
post #7

Earlier quoted context omitted.

Well, as companies get larger they can afford less and less to make mistakes. If I'm running a start-up company, I can get away with changing things abruptly, because I have a smaller base of users and I have fewer people relying on my running at production-level. Also, chances are I'll be able to look at user feedback effectively. Once you get large, and your company is providing sustenance for all your employees, p…

This thinking has always kind of confused me. Why are customers/users at a big company more important than those at a startup? Just because there's more of them, now you can't make mistakes? If you have the agility to make rapid production changes, you also have the ability to rapidly rollback. So the argument that larger companies require more checks and testing than startups isn't really valid, especially when you…

If you have the agility to make rapid production changes, you also have the ability to rapidly rollback.

This is just not true. Rollbacks are always more expensive than changes, because you can't rewind time to undo the consequences of having your software be broken for minutes, hours, or days. Worse, in the absence of "checks", the cost of making a production change tends to be roughly constant as the company grows -- it takes the Amazon sysadmin no more time to type "make deploy" than it does me -- but the cost of a rollback scales directly with the size of your company's customer base.

Within a few seconds after Amazon.com breaks S3, thousands of companies begin to lose money, and they lose money second by second until the rollback happens. Even if Amazon is only down for a minute, that's one minute of downtime multiplied by its number of customers. The larger the customer base, the larger the stakes.

And, unfortunately, the cost of downtime is nonlinear. If Amazon goes down for a mere two minutes, hundreds of peacefully sleeping system administrators will get emergency pages from their uptime-monitoring systems. They will get out of bed. They will check their logs and their failover mechanisms. They will lose a lot of sleep, and soak up a bunch of overtime pay, and a lot of their good will towards Amazon will dissipate like the morning dew. Once you lose your reputation for quality it takes a lot of work to get it back.

This is why larger companies have more controls. The controls are in place to try and pass the ever-increasing cost of a rollback back to the team that causes the rollbacks. The reason it seems so gosh-darned expensive to add a trivial feature to your flagship app is that it is expensive: If the average rollback costs $1m in revenue and every new feature is only 95% reliable, every new feature costs the company $50k to deploy.

The secret here is: If you want to deploy changes rapidly, don't work on a product that has a lot of uptime-sensitive customers! Start a different product line, or start a beta program, or found a smaller company.

Re: The other half of "Artists Ship"

#18
post #10

Earlier quoted context omitted.

This thinking has always kind of confused me. Why are customers/users at a big company more important than those at a startup? Just because there's more of them, now you can't make mistakes? If you have the agility to make rapid production changes, you also have the ability to rapidly rollback. So the argument that larger companies require more checks and testing than startups isn't really valid, especially when you…

If you have the agility to make rapid production changes, you also have the ability to rapidly rollback. This is just not true. Rollbacks are always more expensive than changes, because you can't rewind time to undo the consequences of having your software be broken for minutes, hours, or days. Worse, in the absence of "checks", the cost of making a production change tends to be roughly constant as the company grows…

S3 is a really bad example because they provide infrastructure. Their customers actually see their entire site go down. Those kinds of companies are the exception. I hope Heroku has rigorous testing and scrutinizes every change, even though they are a startup.

Let's say I own a video site and I want to add threaded comments. If I have 5 users and the site goes down for 5 minutes, those 5 users will get 5 minutes each of annoyance. If I have a million users, each of those users will get 5 minutes of annoyance each also. There is no difference to the user there. So, by adding more checks to make sure the site doesn't go down for 5 minutes when you have more users, you're saying the more users you have, more the important each user becomes. I think that's a strange way of thinking.

(The same is true here of an infrastructure service-- if S3 had 5 users and were more cavalier about their release schedule and broke something, those 5 users would exact the same net effects of downtime as if S3 had 5 million users.)

The awesome benefit of getting threaded comments developed, tested briefly, and pushed in one evening is worth the risk of 5 minutes of downtime compared to the 2 weeks of rigorous testing and approval-by-committee. No matter how many users you have.

Re: The other half of "Artists Ship"

#20
post #15
post #10

Earlier quoted context omitted.

This thinking has always kind of confused me. Why are customers/users at a big company more important than those at a startup? Just because there's more of them, now you can't make mistakes? If you have the agility to make rapid production changes, you also have the ability to rapidly rollback. So the argument that larger companies require more checks and testing than startups isn't really valid, especially when you…

No, it's that most people don't want to use a product that changes often, if at all. They get it for what it is and if it changes, it means disrupting their way of using it. When you're working with a smaller product, you can iterate more and work closely with the people using it. When it gets larger, any change you make will alienate some users, so you have to plan to minimize that alienation. You can change and rol…

I think this is particularly important given that often the biggest cost in using a new product\tool is learning to use it well... if it's changing all the time, with no release schedule or no attempt to stabilize the main release via betas\QA\etc, then it gets very frustrating to learn.
Post reply on HN