Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is something bugging you?

antithesis.com

Post reply on HN
Print view View original

121–130 of 438 posts

Re: Is something bugging you?

#121

Post by AlotOfReading »

coldtea wrote: post #46
llm_trw wrote: post #12

Earlier quoted context omitted.

I have proved my code has no bugs according to the spec. I do not make the claim my spec has no bugs.

With formal proof systems, you can also claim that for your spec.

I've written formal proofs with bugs more than once. Reality is much messier than you can encode into any proof and there will ultimately be a boundary where the real systems you're trying to build can still have bugs.

Formal verification is incredibly, amazingly good if you achieve it, but it's not the same as "perfect".

Re: Is something bugging you?

#122

Post by vdqtp3 »

FirmwareBurner wrote: post #72
jollyllama wrote: post #57

Earlier quoted context omitted.

That rings true and is probably why the 10x engineers I have seen usually work on devops or modify the framework the other devs are using in some way. For example, an engineer who speeds up a build or test suite by an order of magnitude is easily a 10x engineer in most organizations, in terms of man hours saved.

> For example, an engineer who speeds up a build or test suite by an order of magnitude is easily a 10x engineer in most organizations, in terms of man hours saved. Yeah but this isn't something scalable that can happen regularly as part of your job description. Like most jobs/companies don't have so many low hanging fruits to pick that someone can speed of build by orders of magnitude on a weekly basis. It's usually…

> Like most jobs/companies don't have so many low hanging fruits to pick that someone can speed of build by orders of magnitude on a weekly basis

You and I have worked at very different organizations. Everywhere I've been has had insane levels of inefficiency in literally every process.

Re: Is something bugging you?

#123

Post by wwilson »

dkyc wrote: post #85

On mobile, the "Let's talk" button in the top right corner is cut off by the carousel menu overlay. Seems like CSS is still out of scope of the bug fixing magic for now. On a more serious note, it's an interesting blog post, but it comes off as veeery confident about what is clearly an incredibly broad and complex topic. Curious to see how it will work in production.

Aww... crap, you're right. I knew we should have finished the UI testing product and run it on ourselves before launching.

Disclosure: Antithesis co-founder.

Re: Is something bugging you?

#124

Post by rkangel »

shermantanktop wrote: post #82

I kept cringing when I read the words “no bugs.” This is hubris in the classic style - it’s asking for a literal thunderbolt from the heavens. It may be true, but…come on. Everyone who has ever written a program has thought they were done only to find one more bug. It’s the fundamental experience of programming to asymptotically approach zero bugs but never actually get there. Again, perhaps the claim is true but it…

I think there is something interesting about the fact that someone writing "no bugs" makes us all uncomfortable.

If they really did have a complex product, running in production from a sizeable userbase and had 2 bug reports ever, then I think it's a reasonable thing to say.

The fact that it isn't a reasonable thing to say for the most other software is a little sad.

Re: Is something bugging you?

#127

Post by pipo234 »

repelsteeltje wrote: post #110
adamauckland wrote: post #101

Earlier quoted context omitted.

I consider a "bug" to be "it was supposed to do something and failed". Issues around business logic are not failures of the system, the system worked to spec, the spec was not comprehensive enough and now we iterate.

...And now we could probably start debating your narrow definition of "system". ;-)

Most of the software I've built doesn't have "a spec.", but let me zoom in on specs. around streaming media. MPEG DASH, CMAF or even the base media file format (ISO/IEC 14496-12) at times can be pretty vague. In practice, this frequently turns up in actual interoperability issues where it's pretty difficult to point out which of two products is according to spec and which one has a bug.

So yes, I totally agree with GP and would actually go further: a phrase like "we found all the bugs in the database" is nonsense and makes the article less credible.

Re: Is something bugging you?

#128

Post by 0xbadcafebee »

Communitivity wrote: post #106

There are situations where no bugs is an important requirement, if it means no bugs that cause a noticeable failure. Things such as planes, submarines, nuclear reactors. For those there is provably correct code. That takes a long time to write, and I mean a really long time. Applying that to all software doesn't make sense from a commercial perspective. There are areas where improvements can have a big impact though,…

A lot of people underestimate the power of QA. Yeah, it would be great if we could just perfectly engineer something out of the gate. But you can also just take several months to stare at something, poke at it, jiggle it, and fix every conceivable problem, before shipping it. Heresy in the software world, but in every other part of the world it's called quality.

Re: Is something bugging you?

#129

Post by shermantanktop »

rkangel wrote: post #124
shermantanktop wrote: post #82

I kept cringing when I read the words “no bugs.” This is hubris in the classic style - it’s asking for a literal thunderbolt from the heavens. It may be true, but…come on. Everyone who has ever written a program has thought they were done only to find one more bug. It’s the fundamental experience of programming to asymptotically approach zero bugs but never actually get there. Again, perhaps the claim is true but it…

I think there is something interesting about the fact that someone writing "no bugs" makes us all uncomfortable. If they really did have a complex product, running in production from a sizeable userbase and had 2 bug reports ever, then I think it's a reasonable thing to say. The fact that it isn't a reasonable thing to say for the most other software is a little sad.

Right, the claim may be true, but I have a visceral reaction to it. And tbh I'd be hesitant to work with someone who made a zero-bugs claim about their own work.

Re: Is something bugging you?

#130

Post by jldugger »

wwilson wrote: post #95
foobarqux wrote: post #73

Earlier quoted context omitted.

Except it doesn't actually explain in what it does: Is it fuzzing? Do you supply your own test cases? Is it testing hardware non-determinism?

Post author here. Sorry it was vague, but there's only so much detail you can go into in a blog post aimed at general audiences. Our documentation ( https://antithesis.com/docs/ ) has a lot more info. Here's my attempt at a more complete answer: think of the story of the blind men and the elephant. There's a thing, called fuzzing, invented by security researchers. There's a thing, called property-based testing, inven…

I remember watching the Strange Loop video on your testing strategy, and now I need to go back and relearn how it differed from model checking (ie Promela or TLA+). Model checking is probably the big QA story that tech companies ignore because it requires dramatically more education, especially from QA departments typically seen as "inferior" to SWE.
Post reply on HN