131–140 of 179 posts
The reality is the opposite of this post. LLMs are great at rapidly creating rough drafts, and humans are best (when properly trained) at critiquing LLM results. So, LLMs are in fact better at blue-teaming, and humans are better at red-teaming.
Earlier quoted context omitted.
What kind of bugs do you find this way, besides missing sanitization?
You can use the fuzzer to generate test cases instead of writing test cases manually. For example you can make it generate queries and data for a database and generate a list of operations and timings for the operations. Then you can mix assertions into the test so you make sure everything is going as expected. This is very useful because there can be many combinations of inputs and timings etc. and it tests basicall…
Earlier quoted context omitted.
What kind of bugs do you find this way, besides missing sanitization?
You can use the fuzzer to generate test cases instead of writing test cases manually. For example you can make it generate queries and data for a database and generate a list of operations and timings for the operations. Then you can mix assertions into the test so you make sure everything is going as expected. This is very useful because there can be many combinations of inputs and timings etc. and it tests basicall…
Earlier quoted context omitted.
What kind of bugs do you find this way, besides missing sanitization?
You can often find memory errors not directly related to string handling with fuzz testing. More generally, if your program embodies any kind of state machine, you may find that a good fuzzer drives it into states that you did not think should exist.
Earlier quoted context omitted.
I would add that few things slow developer velocity as much as a large suite of comprehensive and brittle tests. This is just as true on greenfield as on legacy. Anticipating future responses: yes, a robust test harness allows you to make changes fearlessly. But most big test suites I’ve seen are less “harness” and more “straight-jacket”
I think a problem with AI productivity metrics is that a lot of the productivity is made up. Most enterprise code involves layers of interfaces. So implementing any feature requires updating 5 layers and mocking + unit testing at each layer. When people say “AI helps me generate tests”, I find that this is what they are usually referring to. Generating hundreds of lines of mock and fake data boilerplate in a few minu…
Something that crosses my mind is if AI generating tests necessitates that it only generates tests with fakes and stubs that exercise no actual logic, the expertise required to notice that, and if it is correctable.
Yesterday, I was working on some OAuth flow stuff. Without replayed responses, I'm not quite sure how I'd test it without writing my own server, and I'm not sure how I'd develop the expertise to do that without, effectively, just returning the responses I expected.
It reminds me that if I eschewed tests with fakes and stubs as untrustworthy in toto, I'd be throwing the baby with the bathwater.
Earlier quoted context omitted.
> You need to constantly check that your documentation is aligned to your specs Documentation, tests, and specs are all ultimately different words for the same thing. You do have to check that your implementation and documentation/spec/tests are aligned, which can be a lot of work if you do so by hand, but that's why we invented automatic methods. Formal verification is theoretically best (that we know of) at this, b…
> different words for the same thing You're still misunderstanding and missing the layer of abstraction, which is what I'm (and others are) talking about We have 3 objects: doc, test, spec. How do you prove they are the same thing? You are arguing that they all point to the same address. I'm arguing they all have the same parent. I think it's pretty trivial to show that they aren't identical, so I'll give two example…
You... don't? There is nothing good that can come from trying to understand crazy. Best to run away as fast as possible if you ever encounter this.
> You are arguing that they all point to the same address.
Oh? I did say if you document something the same way three different times (even if you give each time a different name, as if that somehow makes a difference), you are going to pointlessly end up with the same thing. I am not sure that necessarily equates to "the same address". In fact,
> I'm arguing they all have the same parent.
I also said that if they don't end up being equivalent documentation then you will only find difference in information that isn't useful. And that often that information becomes detrimental (see some of the adjacent comments that go into that problem). This is "having the same parent".
In reality, I "argued" both. You'd have better luck if you read the comments before replying.
> you should simplify things as much as possible but be careful to not simplify further
Exactly. Writing tests, documentation, or specs (whatever you want to call it; it all caries the same intent) in natural language certainly feels simpler in the moment, but you'll pay the price later. In reality, you at very least need a tool that supports automatic verification. That could mean formal verification, but, as before it's a beast that is tough to wrangle. More realistically, tests are going to be the best choice amid all the tradeoffs. Industry (including the Haskell fanbois, even) have settled on it for good reason.
> docs are written in natural language, tests are written in programming languages.
Technically "docs" is a concept of less specificity. Documentation can be written in natural language, that is true, but it can also be written in code (like what we call tests), or even pictures or video. "Tests" carries more specificity, being a particular way to write documentation — but ultimately they are the same thing. Same goes for "spec". It describes a different level of specificity (less specific than "tests", but more specific than "docs"), but not something entirely different. It is all documentation.
Earlier quoted context omitted.
You can often find memory errors not directly related to string handling with fuzz testing. More generally, if your program embodies any kind of state machine, you may find that a good fuzzer drives it into states that you did not think should exist.
That sounds a bit like using a jackhammer to drive in a nail. Wouldn’t it be smarter to enumerate edge cases and test all permutations of those?
With fuzzing you can randomly generate bad input that passes all of your test cases that were written using by whatever method you have already been using but still causes the application to crash or behave badly. This may mean that there are more tests that you could write that would catch the issue related to the fuzz case, or the fuzz case itself could be used as a test.
Using probability you can get to 90 or 99% or 99.999% or whatever confidence level you need that the software is unaffected by bugs based on the input size / number of fuzz test cases. In many non-critical situations the goal may not be 100% but 'statistically very unlikely with a known probability and error'
Earlier quoted context omitted.
Tao is talking about systems , which are self-sustaining dynamic networks that function independently of who the individual actors and organizations within the system are. You can break up the monopoly at the heart of the blue team system (as the U.S. did with Standard Oil and AT&T) and it will just reform through mergers over generations (as it largely has with Exxon Mobil and Verizon). You can fire or kill all the…
> and it will just reform through mergers over generations You say "just" as though this is a failure of the system, but this is the system working as designed. Economies of scale are half the reason to bother with large-scale enterprise, so they inevitably consolidate to the point of monopoly, so disrupting that monopoly by force to keep the market aligned is an ongoing and never-ending process that you should expec…
This red vs blue team is a good way to understand the capabilities and current utility of LLMs for expert use. I trust them to add tests almost indiscriminately because tests are usually cheap; if they are wrong it’s easy to remove or modify them; and if they are correct, they adds value. But often they don’t test the core functionality; the best tests I still have to write myself. Having LLMs fix bugs or add feature…
so we've reinvented GAN but with LLMs