Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is something bugging you?

antithesis.com

Post reply on HN
Print view View original

51–60 of 438 posts

Re: Is something bugging you?

#51

Post by xyzelement »

I appreciated this post. Separately from what they are talking about, I found this bit insightful:

// This limits the value of testing, because if you had the foresight to write a test for a particular case, then you probably had the foresight to make the code handle that case too.

I often felt this way when I saw developers feel a sense of doing good work and creating safe software because they wrote unit tests like expect add(2,2) = 4. There is basically a 1-1 correlation between cases you thought to test and that you coded for, which is really no better off in terms of unexplored scenarios.

I get that this has some incremental value in catching blatant miscoding and regressions down the road so it's helpful, it's just not getting at the main thing that will kill you.

I felt similarly about human QA back in my finance days that asked developers for a test plan. If the dev writes a test plan, it also only covers what the dev already thought about. So I asked my team to write the vaguest/highest level test plan possible (eg, "it should now be possible to trade a Singaporean bond" rather than "type the Singaporean bond ticker into the field, type the amount, type the yield, click buy or sell") - the vagueness made more room for the QA person to do something different (even things like tabbing vs clicking, or filling the fields out of sequence, or misreading the labels) than how the dev saw it, which is the whole point.

Re: Is something bugging you?

#52

Post by [deleted] »

Rygian wrote: post #4

The writing is really enjoyable. > Programming in this state is like living life surrounded by a force field that protects you from all harm. [...] We deleted all of our dependencies (including Zookeeper) because they had bugs, and wrote our own Paxos implementation in very little time and it _had no bugs_. Being able to make that statement and back it by evidence must be indeed a cool thing.

[deleted]

Re: Is something bugging you?

#53

Post by jwr »

FoundationDB is an impressive achievement, quite possibly the only distributed database out there that lives up to its strict serializability claims (see https://jepsen.io/consistency/models/strict-serializable for a good definition). The way they wrote it is indeed very interesting and a tool that does this for other systems is immediately worth looking at.

Re: Is something bugging you?

#55

Post by jitl »

rvnx wrote: post #32
jitl wrote: post #27

To me this is very reminiscent of time travel debugging tools like the one used for Firefox’s C++ code, rr / Pernosco: https://pernos.co/

Seems more like a fuzzer for Docker images. Like this: https://docs.gitlab.com/ee/user/application_security/coverag... It won't tell you whether the software works correctly, it will just tell you if it raises an exception or crashes. Put a fuzzer on Chrome for example, you won't catch most of the issues it has, though Chrome actually has tons of bugs and issues, but you may find security issues if you devote a big e…

The similarity is about obtaining determinism through something like a hypervisor. The way rr works is it basically writes down the result of all the system calls, etc, basically everything that ended up on the Turing machine’s tape, so you can rewind and replay.

Re: Is something bugging you?

#56

Post by samsquire »

This is really exciting.

I am an absolute beginner at TLA+ but I really like this possible design space.

I have an idea for a package manager that combines type system with this style of deterministic testing and state space exploration.

Imagine knowing that your invocation of

   package-manager install
Will always work because file system and OS state are part of the deterministic model.

or an next gen Helm with type system and state space exploration is tested:

   kubectl apply
will always work when it comes up because all configuration state space exploration has been tested thanks to types.

Re: Is something bugging you?

#57

Post by jollyllama »

FirmwareBurner wrote: post #43
jimbokun wrote: post #35

> The biggest effect was that it gave our tiny engineering team the productivity of a team 50x its size. I feel like the idea of the legendary "10x" developer has been bastardized to just mean workers who work 15 hours a day 6.5 days a week to get something out the door until they burn out. But here's your real 10x (or 50x) productivity. People who implement something very few people even considered or understood to…

Your definition is also vague. Someone still needs to do the legwork. One man armies who can do everything themselves don't really fit in standardized teams where everything is compartmentalized and work divided and spread out. They work best on their own projects with nobody else in their way, no colleagues, no managers, but that's not most jobs. Once you're part of a team, you can't do too much work yourself no mat…

That rings true and is probably why the 10x engineers I have seen usually work on devops or modify the framework the other devs are using in some way. For example, an engineer who speeds up a build or test suite by an order of magnitude is easily a 10x engineer in most organizations, in terms of man hours saved.

Re: Is something bugging you?

#58

Post by [deleted] »

jimbokun wrote: post #35

> The biggest effect was that it gave our tiny engineering team the productivity of a team 50x its size. I feel like the idea of the legendary "10x" developer has been bastardized to just mean workers who work 15 hours a day 6.5 days a week to get something out the door until they burn out. But here's your real 10x (or 50x) productivity. People who implement something very few people even considered or understood to…

[deleted]

Re: Is something bugging you?

#59

Post by bloopernova »

wwilson wrote: post #20
vinnymac wrote: post #17

I wonder if they are working on a time travel debugger. If it is truly deterministic presumably you could visit any point in time after a record is made and replay it.

No comment. :-) Disclosure: I am a co-founder of Antithesis.

It looks amazing, nice work!

Do you have any plans to let small open source teams use the project for free? Obviously you have bills to pay and your customers are happy to do that, but I was wondering if you'd allow open source projects access to your service once a week or something.

Partly because I want to play with this and I can't see my employer or client paying for it! But also it fits neatly into "DX", the Developer Experience, i.e. making the development cycle as friction free for devs as possible. I'm DevOps with a lifelong interest in UX, so DX is something I'm excited about.

Re: Is something bugging you?

#60

Post by samatman »

rvnx wrote: post #48
timwis wrote: post #44

Gosh, I know it's a bit late, but I wish they'd called the product _The Prime Radiant_ Fans of Asimov's _Foundation_ series will appreciate the analogue to how this system aims to predict every eventuality based on every possible combination of events, a la psychohistory. P.S. amazing intro post. Can't wait to try the product.

It would be the opposite of the product: For a software not interacting with the real world, there is only one possibility for frame N+1, if you know the state of a system. https://en.wikipedia.org/wiki/Determinism PRNG are illusions, just misunderstood by humans.

Did you intend to link to https://en.wikipedia.org/wiki/Deterministic_algorithm?
Post reply on HN