I got really excited about this, and I spent a little time looking through the documentation, but I can't figure out how this is different than randomizing unit tests? It seems if I have a unit test suite already, then that's 99% of the work? Am I misunderstanding? I am drawing my conclusions from reading the Getting Started series of the docs, especially the Workloads section: https://antithesis.com/docs/getting_started/workload.html
Is something bugging you?
171–180 of 438 posts
Re: Is something bugging you?
#172Earlier quoted context omitted.
I'm tired of hearing about 10x engineers. I just want to be a good 1x engineer. Or good at anything in life realy.
The “10x engineer” comes from the observation that there is a 10x difference in productivity between the best and the worst engineers. By saying that you want to be a 1x engineer, you’re saying you want to be the least productive engineer possible. 1x is not the average, 1x is the worst.
Hmm, I never thought of it that way. I just heard 10x employees and fit it to what I knew. Which is that 90% of the work is accomplished by about 10% of workers. The other 90% really only get 10% done. So most developers are somewhere on a scale of 0.1 - 1. With 1 being a totally competent and good developer. The 10x people are just different though, it's like a pro-athlete to a regular player. It's not unique to software development, though it may stand out and be sought after more. I've noticed it in pretty much every industry. Some people are just able to achieve flow state in their work and be vastly more productive than others, be it writing code or laying sod. I don't find that there's a lot of in between 1 and 10 though.
Re: Is something bugging you?
#173> The biggest effect was that it gave our tiny engineering team the productivity of a team 50x its size. I feel like the idea of the legendary "10x" developer has been bastardized to just mean workers who work 15 hours a day 6.5 days a week to get something out the door until they burn out. But here's your real 10x (or 50x) productivity. People who implement something very few people even considered or understood to…
When I was in college, I've met a few people that coded _a lot_ faster than me. Typically, they started since they were 12 instead of 21 (like me). That's how 10x engineers exist, by the time they are 30, they have roughly 20 years of programming experience behind their belt instead of 10.
Also, their professional experience is much greater. Sure, their initial jobs at 15 are the occassional weird gig for the uncle/aunt or cousin/nephew but they get picked up by professional firms at 18 and do a job next to their CS studies.
At least, that's how it used to be. Not sure if this is still happening due to the new job environment, but this was the reality from around 2004 to 2018.
For 10x engineers to exist, all it takes is a few examples. To me, everyone is in agreement that they seem to be rare. I point to a public 10x engineer. He'd never say it himself, but my guess is that this person is a 10x engineer [1].
If you disagree, I'm curious how you'd disagree. I'm just a blind man touching a part of the elephant [2]. I do not claim to see the whole picture.
[1] https://bellard.org/ (the person who created JSLinux)
[2] https://en.wikipedia.org/wiki/Blind_men_and_an_elephant - if you don't know the parable, it's a fun one!
Re: Is something bugging you?
#174This is a great pitch, and I don't want to come across as negative, but I feel like a statement like "we found all bugs" can only be true with a very narrow definition of bug. The most pernicious, hard-to-find bugs that I've come across have all been around the business logic of an application, rather than it hitting into an error state. I'm thinking of the category where you have something like "a database is curren…
I do think that it was a mistake to use the word "all" and imply that there are absolutely no bugs in FoundationDB. However, FoundationDB is truly known as having advanced the state of the art for testing practices: https://apple.github.io/foundationdb/testing.html . So in normal cases this would reek of someone being arrogant / overconfident, but here they really have gotten very close to zero bugs.
The other issue I would point out is that building a database, while impressive with their quality, is still fundamentally different than an application or set of applications like a larger SaaS offering would involve (api, web, mobile, etc). Like the difference between API and UI test strategies, where API has much more clearly defined and standardized inputs and outputs.
To be clear, I am not saying that you can't define all inputs and outputs of a "complete SaaS product offering stack", because you likely could, though if it's already been built by someone that doesn't have these things in mind, then it's a different problem space to find bugs.
As someone who has spent the last 15 years championing quality strategy for companies and training folks of varying roles on how to properly assess risk, it does indeed feel like this has a more narrow scope of "bug" as a definition, in the sort of way that a developer could try to claim that robust unit tests would catch "any" bugs, or even most of them. The types of risk to a software's quality have larger surface areas than at that level.
Re: Is something bugging you?
#175Earlier quoted context omitted.
Post author here. Sorry it was vague, but there's only so much detail you can go into in a blog post aimed at general audiences. Our documentation ( https://antithesis.com/docs/ ) has a lot more info. Here's my attempt at a more complete answer: think of the story of the blind men and the elephant. There's a thing, called fuzzing, invented by security researchers. There's a thing, called property-based testing, inven…
Has any thought been given to repurposing this deterministic computer for more than just autonomous testing/fuzzing? For example, given an ability to record/snapshot the state, resumable software (i.e. durable execution)?
Somebody once suggested to me that this could be very hand for the reproducible builds folks. I'm sure that now that we're out in the open, lots of people will suggest great applications for it.
Disclosure: Antithesis co-founder.
Re: Is something bugging you?
#176Earlier quoted context omitted.
the worst engineer certainly has negative productivity, so I'm not sure that your explanation can possibly be the correct one.
I’m explaining what the terms “10x” and “1x” mean, not asserting that the original observation is correct under all circumstances.
Except you haven't explained it at all. Sackman, Erickson, and Grant found that some developers were able to complete what was effectively a programming contest in a 10th of the time of the slowest participants. This is the origin of the 10x developer idea.
You, on the other hand, are claiming that 10x engineers are 10 times more productive than the worst engineers. Completing a programming challenge in a 10th of the time is not the same as being 10 times more productive, and obviously your usage can't be an explanation, even as one you made up on the spot, as the math doesn't add up.
Re: Is something bugging you?
#177Earlier quoted context omitted.
I'm tired of hearing about 10x engineers. I just want to be a good 1x engineer. Or good at anything in life realy.
The “10x engineer” comes from the observation that there is a 10x difference in productivity between the best and the worst engineers. By saying that you want to be a 1x engineer, you’re saying you want to be the least productive engineer possible. 1x is not the average, 1x is the worst.
Even if this was the origin of the term, it still doesn't make sense because the best engineers can solve problems the worst would never be able to do so. The difference between the best and worst is much more than 10x the worst. Maybe the worst who meets certain minimums at a company, but then the best would also be limited by those willing to work for what the company pays, and I hypothesis that the minimums of the lower bound and the maximums of the upper bound are correlated.
Re: Is something bugging you?
#178Three thoughts: 1. It's a brilliant idea that came at the right time. It feels like people are finally losing patience with flaky software, see developer sentiment on: fuzzers, static typing, memory safety, standardized protocols, containers, etc. 2. It's meant to be niche. $2 per hour per CPU (or $7000 per year per CPU if reserved), no free tier for hobby or FOSS, and the only way to try/buy is to contact them. Ouch…
"It's meant to be niche. $2 per hour per CPU (or $7000 per year per CPU if reserved), no free tier for hobby or FOSS, and the only way to try/buy is to contact them. Ouch. It's a valid business model, I'm just sad it's not going for maximum positive impact."
This is the sort of thing that, if it takes off, will start affecting the entire software world. Hardware will start adding features to support it. In 30 years this may simply be how computing works. But the pioneers need to recover the costs of the arrows they got stuck with before it can really spread out. Don't look at this an event, but as the beginning of a process.
Re: Is something bugging you?
#179Earlier quoted context omitted.
The “10x engineer” comes from the observation that there is a 10x difference in productivity between the best and the worst engineers. By saying that you want to be a 1x engineer, you’re saying you want to be the least productive engineer possible. 1x is not the average, 1x is the worst.
I'm not sure your math works. What we do know is that the worst engineers provide negative productivity. If 1x is the worst engineer, then let's for the sake of discussion denote x as -1 in order for the product to be negative. Except that means the 10x engineer provides -10 productivity, actually making them the worst engineer. Therein lies a conflict. What we also know is that best engineer has positive productivit…
Thank you. This sounds so trivial at first, but your reductio ad absurdum at the beginning of your comment really nails it.
Throw into the mix the fact that productivity is hard to measure as soon as more than one person works on something and that doesn't even begin to consider the economical aspects of software.
And even when ignoring this point, there's that pesky short-term vs long-term thing.
Also, how do you define the term "productivity"? I was assuming that you mean somethint along the lines of (indirect, if employed) monetary output.
Re: Is something bugging you?
#180"I love me a powerful type system, but it’s not the same as actually running your software in thousands and thousands of crazy situations you’d never dreamed of."
Would not trust. Formal software verification is badly needed. Running thousands of tests means almost nothing in software world. Don't fool beginners with your test hero stories.