A three-page paper that shook philosophy, with lessons for software engineers • Hacker News Board

Re: A three-page paper that shook philosophy, with lessons for software engineers

#151

Post by mcgwiz » Tue, Jan 15, 2019, 7:14 PM UTC

I initially thought this article might be about programming by coincidence [1] or maybe about user experience superstitions [2], but after reading it I wonder if this isn't just about the practice of debugging. Software is complex. When someone begins investigation into a bug, if the fix is not immediately found, it becomes a matter of running down a list of hypothetical causes. As one's experience deepens, both the "running down" and the curating of the "list" become more efficient. IMHO this article is merely about a developer who was unaware of certain hypothetical causes.

1: https://pragprog.com/the-pragmatic-programmer/extracts/coinc...

2: https://www.nngroup.com/articles/quality-assurance-ux/

Re: A three-page paper that shook philosophy, with lessons for software engineers

#152

Post by joe_the_user » Tue, Jan 15, 2019, 7:20 PM UTC

I know mathematical logic but I don't know much about conventional philosophy. The Gettier argument seems to indicate that a system of false propositions used to arrive at a conclusion could be called "justification" in normal epistemology - that seems a bit disturbing. Being "justified" in believing X means just having some reason, maybe utterly bogus but existing and convincing to you, for believing X (and then X becomes knowledge even if X happens to be true entirely different reasons). How could no one have commented previously if that was how the system was?

Edit: the main practical implication of the argument seems to be that one cannot assume that when you have argument for X and when you then you get empirical evidence for X, you cannot take that a proof your argument for X is true. It might be suggestive of the truth of the argument but the structure of the argument also has to be taken into account. But that's been a given scientific and statistical investigations for a long time.

Re: A three-page paper that shook philosophy, with lessons for software engineers

#153

Post by isacikgoz » Tue, Jan 15, 2019, 7:25 PM UTC

The article was great. I also find philosophy and software correlates occasionally. The response of the author to the gettier cases are expected behavior. In fact it is a blessing. 99% of the time there is a cow, we experienced that before and that’s the reason of our confidence. We easily solve our problems with this approach.

Re: A three-page paper that shook philosophy, with lessons for software engineers

#154

Post by healsjnr1 » Tue, Jan 15, 2019, 7:31 PM UTC

bunderbunder wrote: post #85

> A philosopher might say that these aren’t bona fide Gettier cases. True gettiers are rare. But it’s still a useful idea... At least as presented, I see the idea being used to do more harm than good. Take the first example, with the form not autofocusing. We're already not in a Gettier case, because the author didn't have JTB. The belief that he caused the bug obviously wasn't true. But it wasn't justified, either.…

I think sometimes the spirit of these observations can be lost in the specifics.

Maybe this doesn't confirm strictly to the philosophical definition, but as an analogy I find it succinct and useful.

Many times I've fallen in the trap of absolutely _knowing_ in my head the reason things are the way they are, only to find out later I'm completely wrong. In most cases the evidence was right in front of me, but software is hard and complex and it is easy for even the best developers to miss the obvious.

Putting a term around it, having an article like this to discuss with the team, these are all useful for reinforcing the idea that we need to continually challenge or knowledge and question assumptions.

Re: A three-page paper that shook philosophy, with lessons for software engineers

#155

Post by zacherates » Tue, Jan 15, 2019, 7:32 PM UTC

chii wrote: post #27

austinjp wrote: post #2

The "cow in the field" example reminds me of two heuristics I like: am I right for the wrong reason?; am I wrong for the right reason? Being right for the wrong reason is dangerous: it's not easy to spot, and it perpetuates false sense of security leaving "black swan events" unanticipated. This might occur during debugging as the article points out, or e.g. during A/B testing of a product. Bring wrong for the right r…

> Bring wrong for the right reason is just plain frustrating. what's an example of being wrong for the right reason? I can't think of any cases where this happens...

Games are a place this comes of frequently. In poker you will often make the "right" (highest EV) play and still lose a given hand.

Making the highest odds plays (and being able to figure out what they are) over and over again regardless of how the individual hands turn out is how you win.

Obviously, you can also win while making the wrong plays by getting lucky (right for the wrong reasons). Evaluating your play based on the outcome of the hands (did you win or lose) rather than the plays you made with the information you had at the time is called Results Oriented Thinking: https://www.pokerdictionary.net/glossary/results-oriented-th... ... and it is a pernicious mistake (with wider applications than just poker :).

Re: A three-page paper that shook philosophy, with lessons for software engineers

#156

Post by robbrit » Tue, Jan 15, 2019, 7:34 PM UTC

This is precisely why when dealing with bugs I advise juniors to avoid asking the question, "what changed?" Gettier cases are just one problem that you can face when asking that question.

Instead I usually tell them to do it the proper way: start from the bug, and work backwards to understand why that bug is happening. At that point the change that caused the bug becomes obvious, and most of the time we realize that we probably wouldn't have come to that conclusion by looking just at what changed.

Re: A three-page paper that shook philosophy, with lessons for software engineers

#157

Post by clairity » Tue, Jan 15, 2019, 7:45 PM UTC

healsjnr1 wrote: post #154

bunderbunder wrote: post #85

> A philosopher might say that these aren’t bona fide Gettier cases. True gettiers are rare. But it’s still a useful idea... At least as presented, I see the idea being used to do more harm than good. Take the first example, with the form not autofocusing. We're already not in a Gettier case, because the author didn't have JTB. The belief that he caused the bug obviously wasn't true. But it wasn't justified, either.…

I think sometimes the spirit of these observations can be lost in the specifics. Maybe this doesn't confirm strictly to the philosophical definition, but as an analogy I find it succinct and useful. Many times I've fallen in the trap of absolutely _knowing_ in my head the reason things are the way they are, only to find out later I'm completely wrong. In most cases the evidence was right in front of me, but software…

one countermeasure i use for this is to substitute the phrase "i'm pretty sure that..." in place of "i know...". that gives both you and others the mental room to consider alternative causes. the 5 why's is another more formalized technique: https://en.wikipedia.org/wiki/5_Whys

Re: A three-page paper that shook philosophy, with lessons for software engineers

#158

Post by dTal » Tue, Jan 15, 2019, 7:57 PM UTC

thewarrior wrote: post #15

I think it helps to look at the mind as a probabilistic survival engine than some truth engine. If there appears to be a cow in a random field the odds are extremely low that someone put a papier mache cow there. If there’s something that has 50 % chance of being a snake you panic and run because that’s a 50 % chance of dying. In the case of the authors bug yes the change he introduced had a good probability of being…

This is an excellent comment - I'm just disappointed you managed to say it all without the word "Bayesian". The base rate for paper mache cows in fields is very low - one is perfectly justified in assigning a decent probability that a field contains a cow, if one sees a cow-shaped object in it. If you are in a part of the world that has a lot of cows in fields, you will presumably assign an even higher probability. You might even say you're "sure" there's a cow in the field, and act as such for everyday purposes. But don't be fooled - you're not really sure. If someone offers to bet you ten dollars versus your life that there is a cow in the field, you'll likely not take the bet.

It seems that the philosophers were grasping towards a definition of "know" that encapsulated the idea of assigning 100% probability to something, after incorporating all the evidence. From a Bayesian standpoint, this is impossible. You can never be 100% certain of anything. To "know" something in the sense of "a justified, true belief" is impossible, because a belief is never both 100% and justified.

(Note that it is entirely possible to see the paper mache cow and conclude that it is likely only paper mache and that there is not a real cow in the field. Is this belief "justified"?)

Re: A three-page paper that shook philosophy, with lessons for software engineers

#159

Post by mdorazio » Tue, Jan 15, 2019, 8:04 PM UTC

cturner wrote: post #129

mdorazio wrote: post #119

Earlier quoted context omitted.

How is it not a JTB? Belief: The pull request broke the search field auto focus. Truth: The pull request did break it. There was an additional reason beyond the pull request unknown to the author, but that's not important to the Truth portion here. Justified: This is the only one you can really debate on, just as philosophers have for a long time. Was he justified in his belief that he broke autofocus? I think so bas…

Maybe I am arguing the same point as you here, but I am uncomfortable that you are painting the justification criteria as being debatable in these situations. In particular, I think your criteria for justification is too low. The standard for justification is - however much is necessary to close off the possibility of something being not-wrong. I find the JTB concept to be useful to reminder us (1) that the concept o…

> however much is necessary to close off the possibility of something being not-wrong.

This is borderline self-referential with respect to the whole Knowledge definition, though. If you have enough information to remove the possibility of a belief being not-wrong then there's no point in defining Knowledge at all. The whole debate around the definition is that humans have to deal with imperfect information all the time, and deciding what constitutes Knowledge in that environment is a challenge.

Re: A three-page paper that shook philosophy, with lessons for software engineers

#160

Post by mdorazio » Tue, Jan 15, 2019, 8:13 PM UTC

bunderbunder wrote: post #126

mdorazio wrote: post #119

Earlier quoted context omitted.

How is it not a JTB? Belief: The pull request broke the search field auto focus. Truth: The pull request did break it. There was an additional reason beyond the pull request unknown to the author, but that's not important to the Truth portion here. Justified: This is the only one you can really debate on, just as philosophers have for a long time. Was he justified in his belief that he broke autofocus? I think so bas…

His real belief was not exactly that the PR broke it, it's that the root cause of the break was isolated to his code changes. This is evident from the debugging procedure he described. And that distinction is very important, because that detail, and not some abstract piece of philosophy, is also the real source of the challenges that motivated describing the situation in a blog post in the first place. What I'm reall…

I think you're interpreting the author's belief in a way that you want rather than what he actually says. I read the belief based on this statement from the article:

"When I released the new version, I noticed that I’d broken the autofocusing of the search field that was supposed to happen on pageload."

That's it. That's the belief - he broke autofocusing when he released the new version. This was true. The later digging in to find the root cause is merely related to this belief. And yes I agree that Gettier's cases were meant to show that correct belief for the wrong reasons (maintaining the three criteria essentially), but this case meets that intent as well. The author is correct that he broke autofocus via his pull request, and thus JTB holds, but the actual reason for it is not his personal code and thus the Knowledge is incorrect.