Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You should write "without bugs"

korshakov.com

Post reply on HN
Print view View original

81–90 of 91 posts

Re: You should write "without bugs"

#81

Post by atomicnumber3 »

markerz wrote: post #47
wswope wrote: post #17

Earlier quoted context omitted.

> the authors point is that a little bit of extra thought and work can make a lot of difference to quality Care to bring home the thesis on how that’s actually really insightful?

There are two examples that come to mind: I’ve caught multiple production bugs in code review by just carefully reasoning about the code and changing some variable names to match my mental model. I didn’t quite understand how bad it was, but I could tell it wasn’t right so I left a comment saying “this seems wrong”. What happened after? It was merged without addressing and it triggered a P1 regression in production t…

"It just doesn’t really get rewarded."

This is the entirety of the problem. Also why open source programs are so often "surprisingly" high quality.

Bad reward functions in companies don't just not reward people who do good work. It *actively punishes* them because stack ranking is a zero-sum game. And as much as people joke about stack ranking and lambast the dinosaurs who used to do it on purpose, it's still how it all actually work. It's just distributed stack ranking - each manager and manager of managers has their own local stack rank that bubbles up to who gets fired and who gets promoted.

So people who throw shit at the wall and make product that sell but have a shitty user experience get promoted and people who plod along and make things that work, or fix things that are broken (but not so much that they don't sell) filter to the bottom of the list and get cut, or leave when they don't get promoted or get raises.

Sure, there's some golden mix in between throwing shit at the wall and fixing key UX-ruining bugs. But these people still get outcompeted by people who purely ship n scoot.

Re: You should write "without bugs"

#82

Post by alaaalawi »

jp57 wrote: post #35

In grad school I took a formal methods class where we proved properties about programs that completely changed how I think about bugs. The main things I took from the class were 1. Correctness of a program is distinct from its performance. 2. Program correctness can be proven. 3. Optimizing for performance often makes it harder to prove correctness. I do not actually use formal methods in my work as a developer, but…

Achieving Correctness is really satisfying. however it is hard and difficult. IMOH this does in general polarize the scene (proving fanatics on one extreme and the other side who are not even testing the code) IMHO flushing out what you are designing does help and goes along the way of having fewer bugs. one old relatively yet easy and accessible formal toolkit which helpful in flushing out process is Z notation . one of the accessible books, old yet an easy read and rewarding is

"Software Development with Z. A practical approach to formal methods in software engineering"

https://archive.org/details/softwaredevelopm0000word/page/n1... .

there are other notations developed later. but its simplicity and easiness even while scribling on paper or word processor gets me back to using it every now and then

Re: You should write "without bugs"

#83

Post by lcnPylGDnU4H9OF »

Night_Thastus wrote: post #76
bccdee wrote: post #57

Earlier quoted context omitted.

That's a justification for making a product with fewer features, not for making a product that's packed with bugs.

The only product guaranteed to be 'perfect' has 0 features at all.

I think this is over-indexing "perfect" a bit. My intended point is that practicing the "correct" way will lead to improvement and especially to learning "correct" habits. Perhaps replace "correct" with "helpful"; it's about paying attention to the actions one takes and ensuring that they generally aid in progressing towards a goal.

If the goal is to make a program, I make sure that my thoughts about the program are sound before actually building it. If I need to play around a bit to understand the problem domain then I additionally need to be prepared to throw out the play-work. If I run into a bug, I remember it and proactively avoid it in the future. I think about the limitations of my design before I start building. This is my implementation of writing "without bugs"; it doesn't have to be perfect, it just can't be something which I know to be wrong.

Re: You should write "without bugs"

#84

Post by fvdessen »

alaaalawi wrote: post #82
jp57 wrote: post #35

In grad school I took a formal methods class where we proved properties about programs that completely changed how I think about bugs. The main things I took from the class were 1. Correctness of a program is distinct from its performance. 2. Program correctness can be proven. 3. Optimizing for performance often makes it harder to prove correctness. I do not actually use formal methods in my work as a developer, but…

Achieving Correctness is really satisfying. however it is hard and difficult. IMOH this does in general polarize the scene (proving fanatics on one extreme and the other side who are not even testing the code) IMHO flushing out what you are designing does help and goes along the way of having fewer bugs. one old relatively yet easy and accessible formal toolkit which helpful in flushing out process is Z notation . on…

I've a good track record of having my programs work without bugs, I don't think it's too hard. The way I work is to restrict myself to using building blocks that I know work well and produce correct results. For example: using state machines, never breaking out of a loop, tackling all the edge case before the body, using simple data structures, don't use inheritance or exceptions, don't interact with files natively, don't use recursion, etc. etc.

When I face a programming problem I map the solution to those simple building blocks, and If I can't I try to change the problem first.

Formal methods are hard if you want to prove the correctness of a hard algorithm, but you can almost always do without a hard algorithm and use something really basic instead, and for the basic things you don't need formal methods.

The people who write the most bugs in my experience do it because they don't fully understand the building blocks they're using, and rely on other things for correctness like the type checker or unit tests. They view code as a stochastic process, write code they don't fully understand and have accepted the resulting bugs as a fact of life.

Re: You should write "without bugs"

#85

Post by taeric »

ChrisMarshallNY wrote: post #80
taeric wrote: post #79

Earlier quoted context omitted.

I mean, is that a general feel across industry, or something a rando online offered up as a witty quip? I have never worked anywhere where people were casual about shipping bugs. Some of the healthier places I've been have focused on not getting worked up when you do. Don't take pride in it, but don't sweat the small mistakes. You will make them, whether you want to or not.

I wouldn’t know, but it was certainly presented as a “general feel,” and, anecdotally, almost every interaction that I’ve personally had with “modern” tech companies (in fact, with one, it resulted in a multimillion-dollar disaster), have shown me that “feel.” The disaster I mentioned was particularly heartbreaking, because the tech was solid, and the people behind it, were good, but they absolutely refused to give r…

I have seen people that love to argue the straw man of why 100% code coverage is not something that will often, if ever, give 100% confidence that your product is correct. But I have never met anyone that would not agree that <50% test coverage is a red flag. Could get most people up to about 85% before they would balk at the focus.

Re: You should write "without bugs"

#86

Post by ChrisMarshallNY »

taeric wrote: post #85
ChrisMarshallNY wrote: post #80

Earlier quoted context omitted.

I wouldn’t know, but it was certainly presented as a “general feel,” and, anecdotally, almost every interaction that I’ve personally had with “modern” tech companies (in fact, with one, it resulted in a multimillion-dollar disaster), have shown me that “feel.” The disaster I mentioned was particularly heartbreaking, because the tech was solid, and the people behind it, were good, but they absolutely refused to give r…

I have seen people that love to argue the straw man of why 100% code coverage is not something that will often, if ever, give 100% confidence that your product is correct. But I have never met anyone that would not agree that <50% test coverage is a red flag. Could get most people up to about 85% before they would balk at the focus.

I would take it a bit further: https://littlegreenviper.com/testing-harness-vs-unit/

Re: You should write "without bugs"

#87

Post by eternityforest »

Very interesting! I wonder if this is actually possible or practical for all devs.

Then mention subconscious skill, and I think some people improve with practice more than others, and there may be individual differences in the "Noise floor" of trivial mistakes, like swapping plus and minus for no reason you could explain, they're just in the same category and you vaguely pattern matched that an operator goes there, and didn't think about it because people don't analyze every single token like a robot.

I can ship code that is more or less bug free, and I always aim to do so, but I don't really consider the idea of directly writing good code.

I try to remove opportunities to fail, designing an architecture that I think wouldn't take a lot of skill to implement, and then write the code, and then assume I have still failed in every imaginable way, and check for those failures, even if I have looked over the code ten times and am sure that I didn't make some particular mistake.

This is generally how I think about life in general too, I'm always thinking "Use a protective case" instead of "Just don't drop it".

Re: You should write "without bugs"

#88

Post by AtlasBarfed »

hulitu wrote: post #64
Ferret7446 wrote: post #62

Earlier quoted context omitted.

> Program correctness can be proven. It can be proven, in broadly the same sense as all of the atoms in your body can simultaneously "decide" to exist an inch to the right due to quantum field theory. It is not practical to prove correctness for the vast majority of programs, and there are programs that are demonstrably incorrect, cannot be made correct, and yet are useful and still function anyway (e.g., closing TCP…

> It is not practical to prove correctness for the vast majority of programs, That is the excuse i hear a lot from software developers, when everything they do is to test the expected behaviour of a program, without any edge case. And this is also the reason why iMessage and WhatsUp are full of one click exploits.

Oh yeah, just prove it correct. Simple

Wait, code is not simple. Are the requirements simple? Oh wait, in reality they aren't simple either.

The complexity of the formal specification must ramp up with the complexity of the code, because the task being asked is complex.

So where does the debugging end if the specification is wrong?

How do you prove the requirements are correct?

Unit testing is helpful but wasn't the panacea, because the most important bugs are in integration.

I'm also going to guess that formal proofs will have hellacious issues with system boundaries. They can't prove correctness of the state on the other end of the pipe.

So you overcome that, time and money. What happens when the requirements/specification changes? How much of the verification is invalidated?

Proof systems need to prove their economic validity in the he software realm. Heck, where are the proof systems for subsections of the Linux kernel or other vitally important inner loops of computing? Those actually have stable specifications. Weren't go multitasking formally verified/provably correct in some sense? Why haven't correctness tools tackled other things?

Re: You should write "without bugs"

#89

Post by SAI_Peregrinus »

Jtsummers wrote: post #52
SAI_Peregrinus wrote: post #46

Earlier quoted context omitted.

Correctness of a program is usually distinct from performance. One obvious exception is code processing secret data. There, performance variance creates observable side-effects (timing side channels) which can be used to determine the secrets' values. Another is any sort of hard real-time system, where performance is critical to correctness. For example, a brake-by-wire system that took 10 seconds to respond to the p…

Even for hard real-time systems, focusing on correctness first is often the right way to go. At least in my experience, it's typically been much easier to make a correct system fast than a fast wrong system correct. With some particularly delightful (read: catastrophic) results when some folks really wanted to push their fast wrong code (fixed years ago, fortunately, and they had a good corporate culture change not l…

I agree. But performance can't be ignored, and for some systems like those I mentioned it's not distinct from correctness. Performance doesn't always mean "as fast as possible", e.g. for systems dealing with secret data it means "without leaking information via side channels", for something like petting a window watchdog it means "slow enough not to pet before the window opens, fast enough not to pet after the window closes".

Re: You should write "without bugs"

#90

Post by meoleo »

strken wrote: post #69
hulitu wrote: post #64

Earlier quoted context omitted.

> It is not practical to prove correctness for the vast majority of programs, That is the excuse i hear a lot from software developers, when everything they do is to test the expected behaviour of a program, without any edge case. And this is also the reason why iMessage and WhatsUp are full of one click exploits.

The comment you were replying to is about formal proofs of correctness, probably using tools like Rocq (formerly Coq) and TLA+. These tools can take an extreme amount of time to use and require specialised skillsets that are difficult to hire for. They go far beyond standard software engineering practices like unit and E2E testing. It is genuinely, seriously not practical to verify the average startup's SaaS web app…

TLA+ and Coq are too hard for most SAAS applications. Tried FizzBee (https://fizzbee.io/tutorials/getting-started/) and P (https://p-org.github.io/P/whatisP/)? These seems quite practical.

Coq/Rocq are obviously much harder to use, and I mostly see embedded systems engineers use them. TLA+ is somewhat easier, and being used in a number of cloud infrastructure companies (AWS, MongoDB, Confluent etc).

Obviously, if you are building a simple CRUDL app, these tools don't help at all.

Post reply on HN