Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is something bugging you?

antithesis.com

Post reply on HN
Print view View original

371–380 of 438 posts

Re: Is something bugging you?

#371

Post by maronato »

BoppreH wrote: post #97

Three thoughts: 1. It's a brilliant idea that came at the right time. It feels like people are finally losing patience with flaky software, see developer sentiment on: fuzzers, static typing, memory safety, standardized protocols, containers, etc. 2. It's meant to be niche. $2 per hour per CPU (or $7000 per year per CPU if reserved), no free tier for hobby or FOSS, and the only way to try/buy is to contact them. Ouch…

I think their target audience is teams who already have mature software and comprehensive tests. From the docs, the kinds of bugs their platform is designed to find are the wild “unreproducible” kind that only happens rarely in production. Most teams have much bigger problems and obvious bugs to fix.

Heck, most software in production today barely has unit tests.

Re: Is something bugging you?

#372

Post by maronato »

amw-zero wrote: post #45

I'm trying to avoid diving into the hype cycle about this immediately - but this sounds like the holy grail right? Use your existing application as-is (assuming it's containerized), and simply check properties on it? The blocker in doing that has always been the foundations of our machines: non-deterministic CPUs and operating systems. Re-building an entire vertical computing stack is practically impossible, so they…

You still have to use their SDKs to write lots of integration tests (they call them “workloads”).

Then they run those tests while injecting all sorts of failures like OS failures, network issues, race and timing conditions, random number generator issues, etc.

It’s likely the only practical way today of testing for those things reliably, but you still have to write all of the tests and define your app state.

Re: Is something bugging you?

#373

Post by toast0 »

Voultapher wrote: post #288
indiv0 wrote: post #10

I've been super interested in this field since finding out about it from the `sled` simulation guide [0] (which outlines how FoundationDB does what they do). Currently bringing a similar kind of testing in to our workplace by writing our services to run on top of `madsim` [1]. This lets us continue writing async/await-style services in tokio but then (in tests) replace them with a deterministic executor that patches…

> you can test hundreds of thousands of subtle failure cases in I/O, event ordering, timeouts, dropped packets, filesystem failures, etc. As cool as all this is, I can't stop but wonder how often the culture of micro-services and distributed computing is ill advised. So much complexity I've seen in such systems boils down to calling a "function" is: async, depends on the OS, is executed at some point or never, always…

> I can't stop but wonder how often the culture of micro-services and distributed computing is ill advised.

You can't get away from distributed computing, unless you get away from computing. A modern computer isn't a single unit, it's a system of computers talking to each other. Even if you go back a long time, you'll find many computers or proto-computers talking to each other, but with a lot stricter timings, as the computers are less flexible.

If you save a file to a disk, you're really asking the OS (somehow) to send a message to the computer on the storage device, asking it to store your data, and it will respond with success or failure and it might also write the data. (sometimes it will tell your os success and then proceed to throw the data away, which is always fun)

That said, keeping things together where it makes sense, is definitely a good thing.

Re: Is something bugging you?

#374

Post by chris_wot »

JohnMakin wrote: post #351
sangnoir wrote: post #230

Earlier quoted context omitted.

>It seems like the industry would get a lot more 10x behavior if it was recognized and rewarded more often than it currently does I'd be happier if industry cares more for team productivity - I have witnessed how rewarding "10x" individuals may lead to perverse results on a wider scale, a la Cobra Effect. In one insidious case, our management-enabled, long-tenured "10x" rockstar fixed all the big customer-facing bugs…

> In one insidious case, our management-enabled, long-tenured "10x" rockstar fixed all the big customer-facing bugs quickly, but would create multiple smaller bugs and regressions for the 1x developers to fix while he moved to the next big problem worthy of his attention. Everyone else ended up being 0.7x - which made the curse of an engineer look even more productive comparatively! Because he was allowed to break th…

What happened to him in the end?

Re: Is something bugging you?

#375

Post by cortesoft »

loeg wrote: post #348
wwilson wrote: post #164

Earlier quoted context omitted.

Nope. We’re emulating a deterministic computer, so your software can’t act nondeterministically if it tries.

Do you emit deterministic sequences from things like RDRAND? I guess you'd have to.

Yes, they said they do

Re: Is something bugging you?

#376

Post by cortesoft »

adamauckland wrote: post #101
benrutter wrote: post #98

This is a great pitch, and I don't want to come across as negative, but I feel like a statement like "we found all bugs" can only be true with a very narrow definition of bug. The most pernicious, hard-to-find bugs that I've come across have all been around the business logic of an application, rather than it hitting into an error state. I'm thinking of the category where you have something like "a database is curren…

I consider a "bug" to be "it was supposed to do something and failed". Issues around business logic are not failures of the system, the system worked to spec, the spec was not comprehensive enough and now we iterate.

I agree they are separate, but in my long experience, spec bugs are at least as common as your first definition.

Re: Is something bugging you?

#377

Post by cortesoft »

coldtea wrote: post #46
llm_trw wrote: post #12

Earlier quoted context omitted.

I have proved my code has no bugs according to the spec. I do not make the claim my spec has no bugs.

With formal proof systems, you can also claim that for your spec.

This doesn't track with the real world, though.

If you are writing software, it is almost always trying to accomplish a goal outside of itself. It is trying to solve a problem for someone, and how that problem can or should be solved is rarely perfectly clear.

The spec is supposed to map to a real world problem, and there is never going to be a way to formalize that mapping.

Re: Is something bugging you?

#378

Post by pfdietz »

fleaflicker wrote: post #108

Business value is a good way to think about it: > As a software developer, fixing bugs is a good thing. Right? Isn’t it always a good thing? > No! > Fixing bugs is only important when the value of having the bug fixed exceeds the cost of the fixing it. https://www.joelonsoftware.com/2001/07/31/hard-assed-bug-fix...

If you have high volume automated testing, you want to fix all the bugs, even ones not directly affecting customers, otherwise they keep showing up and may keep you from finding other more customer-relevant bugs.

Re: Is something bugging you?

#379

Post by Voultapher »

toast0 wrote: post #373
Voultapher wrote: post #288

Earlier quoted context omitted.

> you can test hundreds of thousands of subtle failure cases in I/O, event ordering, timeouts, dropped packets, filesystem failures, etc. As cool as all this is, I can't stop but wonder how often the culture of micro-services and distributed computing is ill advised. So much complexity I've seen in such systems boils down to calling a "function" is: async, depends on the OS, is executed at some point or never, always…

> I can't stop but wonder how often the culture of micro-services and distributed computing is ill advised. You can't get away from distributed computing, unless you get away from computing. A modern computer isn't a single unit, it's a system of computers talking to each other. Even if you go back a long time, you'll find many computers or proto-computers talking to each other, but with a lot stricter timings, as th…

I see your point. Even multithreading can be seen as a form of distributed programming. At the same time, in my experience these parts can often be isolated. You trust your DB to handle such issues, and I'm very happy we are getting a new era of DBs like Tigerbetle, FoundationDB and sled that are designed to survive Jepsen. But how many teams are building DBs? That point is a bit ironic, given I'm currently building an in-memory DB at work. But it's a completely different level of complexity. And your example with writing a file, that too is a somewhat solved problem, use ZFS. I'd argue there are many situations where the fault tolerant distributed requirements can be served by existing abstractions.

Re: Is something bugging you?

#380

Post by rapjr9 »

I love this idea. In the early days of computing computers were claimed to always be deterministic. Give it the same inputs and you get the same outputs. Little by little that disappeared, with interrupts, with multithreading, with human derived inputs, with multitasking, with distributed processing, until today computers and applications are often not deterministic at all, and it does indeed make them very difficult to test. Bringing back the determinism may not only be good for testing, it seems likely to improve reliability. While I see how this is great for distributed databases I wonder if it has application when inputs are inherently non-deterministic (e.g., human input, sensor derived inputs).
Post reply on HN