The economics of software correctness • Hacker News Board

Re: The economics of software correctness

#51

Post by obastani » Tue, Oct 06, 2015, 9:56 PM UTC

DRMacIver wrote: post #47

obastani wrote: post #23

Earlier quoted context omitted.

I don't think that's a fair analogy. At this point, I believe we have pretty good ideas that make it possible to write correct software. Most impressively, there has been progress on creating a completely verified C compiler named CompCert: http://compcert.inria.fr/ . This really means "completely verified"---they formalized the semantics both of C and of assembly, and used an automated theorem to prove that the sema…

> I don't think that's a fair analogy. At this point, I believe we have pretty good ideas that make it possible to write correct software. Most impressively, there has been progress on creating a completely verified C compiler named CompCert: http://compcert.inria.fr/ . This really means "completely verified"---they formalized the semantics both of C and of assembly, and used an automated theorem to prove that the se…

Yes, I was referring to the verified parts. To the best of my knowledge, these are bug free -- I believe the bugs were found in an unverified portion of the front-end (which I think was subsequently verified). I think the technique works in principle; it's just very, very expensive.

Re: The economics of software correctness

#52

Post by mreiland » Tue, Oct 06, 2015, 11:56 PM UTC

nine_k wrote: post #49

mreiland wrote: post #44

Earlier quoted context omitted.

I personally hate working with people who put things in terms of 'right' and 'wrong', or 'correct' and 'incorrect'. I honestly believe the key to software development is good decision making. If you're thinking in terms of right and wrong, correct and incorrect, then you're not making decisions, you're simply doing the right thing over and over again. Only it may not be the OPTIMAL thing, or the smart thing, in any s…

I like the idea of finding a good decision in the context of a current situation. I don't like the idea that terms like 'correct' and 'incorrect' should not be used. For some problems, there exist exactly correct solutions (like adding integers). The lack of a well-founded or at least well-known correct solution is what forces you to look for an optimal solution made of trade-offs.

If you read back over the the thread it will quickly become apparent that your example is not relevant within the context of this conversation.

Re: The economics of software correctness

#53

Post by jaggederest » Wed, Oct 07, 2015, 12:14 PM UTC

nostrademons wrote: post #31

jaggederest wrote: post #4

I think one of the other problems is that people value the economics of software correctness using their gut, rather than empirical analysis. https://en.wikipedia.org/wiki/Hyperbolic_discounting Everyone knows that bugs are problematic eventually, it just seems that they can't put that on a level playing field with the up front costs, be they in terms of time, features, or effectiveness. As an example, if you asked H…

Hyperbolic discounting is rational when the availability of resources increases exponentially, as it often does for products that catch on. A company that comes to market with a product that is useful but buggy will attract the attention of venture capitalists & other investors. It will receive user feedback from its existing user base. It will find it easier to hire top talent. It will be able to use collected data…

I see it continue long past the point where 'we needed to ship something to be able to eat next month', to the point where people handicap themselves for the next 5 years in a mature business to release a feature a month faster even though they've got 3 years of runway and ample revenue.

Also, hyperbolic discounting is explicitly not exponential in the way you might account for the future availability of resources (even at a very high exponential rate), it's valuing things on a different curve in the future than the present: would you rather have five dollars today or ten dollars next month, vs would you rather have five dollars a year from now or ten dollars a year and a month from now. People will say five dollars today, but ten dollars a year and a month from now, even though under rational analysis they should come out exactly the same.

Re: The economics of software correctness

#54

Post by kybernetikos » Wed, Oct 07, 2015, 8:33 PM UTC

mreiland wrote: post #44

kybernetikos wrote: post #14

So this is all true, however it is surprising how often in development you find people spending more time and effort getting the 'pragmatic' hack to work acceptably than it would have taken to do it right. People often underestimate the difficulties of getting a hack to work and overestimate how long doing something correctly will take. Even with short time horizons.

I personally hate working with people who put things in terms of 'right' and 'wrong', or 'correct' and 'incorrect'. I honestly believe the key to software development is good decision making. If you're thinking in terms of right and wrong, correct and incorrect, then you're not making decisions, you're simply doing the right thing over and over again. Only it may not be the OPTIMAL thing, or the smart thing, in any s…

> I honestly believe the key to software development is good decision making.

I agree, and the purpose of my comment was to point out that I have often observed people making decisions on a case by case basis, and repeatedly making bad decisions by underestimating how much effort the solution thought to be 'quicker' or 'easier' or 'minimal change' option is compared to the solution thought to be more 'radical' or 'unnecessary effort'.

You seem to have some specific bone to pick around testing. I'm not really interested in that. When I say 'right' or 'correct' in this context, it's generally about not being blinkered by the current state of your codebase. People get so used to the fact that their current codebase does things in a particular way, they often look at problems as situations where they have no choice but to force the problem into the frame of their code. Solutions where you solve the problem with something closer to what you would want to write if you didn't have to worry quite so much about the peculiarities of your current codebase generally scare people, but are often the better decision not just from some academic point of view, but literally from an execution, getting-things-done quickly point of view. 'correct' as I use it in this context is mainly a shorthand for that concept.

The whole point here is not to say that there's some secret that avoids you from having to make decisions, it's to say that many people in the real world are making a particular class of error in decision making. Signs that a particular decision might fall in that category include:

1. the main trade off being considered between the options is developer time

2. none of the developers actually think the believed-to-be-quicker-to-implement option is a good option apart from its supposed quickness to implement.

3. other options exist where a significant fraction of developers agree that apart from their supposed slowness to implement, they are good options.