I love the idea, but this line: > 1) no bug should take over 2 days Is odd. It’s virtually impossible for me to estimate how long it will take to fix a bug, until the job is done. That said, unless fixing a bug requires a significant refactor/rewrite, I can’t imagine spending more than a day on one. Also, I tend to attack bugs by priority/severity, as opposed to difficulty. Some of the most serious bugs are often qui…
Bugs taking less than 2 days are great to have as a target but will not be something that can be guaranteed.
We stopped roadmap work for a week and fixed bugs
81–90 of 356 posts
Re: We stopped roadmap work for a week and fixed bugs
#82Earlier quoted context omitted.
At Amazon we had a bug that was the result of a compiler bug and the behaviour of intel cores being mis-documented. It was intermittent and related to one core occasionally being allowed to access stale data in the cache. We debugged it with a logic analyzer, the commented nginx source and a copy of the C++ 11 spec. It took longer than 2 days to fix.
I’m old enough to have used ICEs to trace program execution. They were damn cool. I seriously doubt that something like that, exists outside of a TSMC or Intel lab, these days.
Re: We stopped roadmap work for a week and fixed bugs
#83I'm a bit torn on Fix-it weeks. They are nice but many bugs simply aren't worth fixing. Generally, if they were worth fixing - they would have been fixed. I do appreciate though that certain people, often very good detail oriented engineers, find large backlogs incredibly frustrating so I support fix-it weeks even if there isn't clear business ROI.
> Generally, if they were worth fixing - they would have been fixed. ??? Basically any major software product accumulates a few issues over time. There's always a "we can fix that later" mindset and it all piles up. MacOS and Windows are both buggy messes. I think I speak for the vast majority of people when I say that I'd prefer they have a fix-it year and just get rid of all the issues instead of trying to rush new…
Re: We stopped roadmap work for a week and fixed bugs
#84It's fairly telling of the state of the software industry that the exotic craft of 'fixing bugs' is apparently worth a LinkedIn-style self-promotional blog post. I don't mean to be too harsh on the author. They mean well. But I am saddened by the wider context, where a dev posts 'we fix bugs occasionally' and everyone is thrilled, because the idea of ensuring software continues to work well over time is now as alien…
We as industry have taught people that broken products is acceptable.
In any other industry, unless people are from the start getting something they know is broken or low quality, flea market, 1 euro shop, or similar, they will return the product, ask for the money back, sue the company whatever.
Re: We stopped roadmap work for a week and fixed bugs
#85Doing what you want to do instead of what you should doing (hint: you should be busy making money).
Inability to triage and live with imperfections.
Not prioritizing business and democratizing decision making.
Re: We stopped roadmap work for a week and fixed bugs
#86It's fairly telling of the state of the software industry that the exotic craft of 'fixing bugs' is apparently worth a LinkedIn-style self-promotional blog post. I don't mean to be too harsh on the author. They mean well. But I am saddened by the wider context, where a dev posts 'we fix bugs occasionally' and everyone is thrilled, because the idea of ensuring software continues to work well over time is now as alien…
This is not the vibe I got from the post at all. I am sure they fix plenty of bugs throughout the rest of the year, but this will be balanced with other work on new features and the like and is going to be guided by wider businesses priorities. It seems the point in the exercise is focusing solely on bugs to the exclusion of everything else, and a lot of latitude to just pick whatever has been annoying you personally.
Re: We stopped roadmap work for a week and fixed bugs
#87False sense of accomplishment. Doing what you want to do instead of what you should doing (hint: you should be busy making money). Inability to triage and live with imperfections. Not prioritizing business and democratizing decision making.
Re: We stopped roadmap work for a week and fixed bugs
#88Earlier quoted context omitted.
> Generally, if they were worth fixing - they would have been fixed. ??? Basically any major software product accumulates a few issues over time. There's always a "we can fix that later" mindset and it all piles up. MacOS and Windows are both buggy messes. I think I speak for the vast majority of people when I say that I'd prefer they have a fix-it year and just get rid of all the issues instead of trying to rush new…
Oh boy, I’d trade one(or easily 2/3) major MacOs version for a year worth of bug fixes in a heartbeat.
>For iOS 27 and next year’s other major operating system updates — including macOS 27 — the company is focused on improving the software’s quality and underlying performance.
-via Bloomberg today
Re: We stopped roadmap work for a week and fixed bugs
#89Earlier quoted context omitted.
My team once encountered a bug that was due to a supplier misstating the delay timing needed for a memory chip. The timings we had in place worked, for most chips, but they failed for a small % of chips in the field. The failure was always exactly identical, the same memory address for corrupted, so it looked exactly like an invalid pointer access. It took multiple engineers months of investigating to finally track d…
But what was the original estimate? And even so I'm not saying it must be completely and always correct. I'm saying it seems wild to have no starting point, to simply give up.
Best case you trap on memory access to an address if your debugger supports it (ours didn't). Worst case you go through every pointer that is known to access nearby memory and go over the code very very carefully.
Of course it doesn't have to be a nearby pointer, it can be any pointer anywhere in the code base causing the problem, you just hope it is a nearby pointer because the alternative is a needle in a haystack.
I forget how we did find the root cause, I think someone may have just guessed bit flip in a pointer (vs overrun) and then un-bit-flipped every one of the possible bits one by one (not that many, only a few MB of memory so not many active bits for pointers...) and seen what was nearby (figuring what the originally intended address of the pointer was) and started investigating what pointer it was originally supposed to be.
Then after confirming it was a bit flip you have to figure out why the hell a subset of your devices are reliably seeing the exact same bit flipped, once every few days.
So to answer your question, you get a bug (memory is being corrupted), you do an initial investigation, and then provide an estimate. That estimate can very well be "no way to tell".
The principal engineer on this particular project (Microsoft Band) had a strict 0 user impacting bugs rule. Accordingly, after one of my guys spend a couple weeks investigating, the principal engineer assigned one of the top firmware engineers in the world to track down this one bug and fix it. It took over a month.
Re: We stopped roadmap work for a week and fixed bugs
#90Earlier quoted context omitted.
You haven't seen the same kind of bugs I have, I guess. This kind of thing takes more than 2 days to fix, unless you're really good. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217637 Or this one https://security.stackexchange.com/questions/104845/dhe-rsa-... I can find more of these that I've run into if I look. I've had tricky bugs in my team's code too, but those don't result in public artifacts, and I'm res…
That's what I'm saying. Nothing takes 2 days to fix meaning it takes more time
Or just an hour or two. I can't find it anymore, but I've run into libraries where simple things with months didn't work, because like May only has three letters or July and June both start with Ju. That can turn into a big deal, but often it's easy, once someone notices it.