Earlier quoted context omitted.
> I understand the argument of "people are not actively looking", but isn't the cost for such a scan getting lower by the week, and Anthropic's Project Glasswing is supposed to find them quite a while ago? You have to consider that having an LLM scan for vulnerabilities is hardly infallible. It is a search guided by heuristics and given a large enough codebase, it is unlikely to identify all vulnerabilities. Personal…
> [A]ll of them found some that the others hadn't discovered. Now, correctness issues aren't the same as vulnerabilities, but the same principle about using heuristics to find defects applies. This makes perfect sense, but that conflicts with the impression put forward by Anthropic and OpenAI (in particular) that they alone occupy 'frontier model' spots. Frontier models should large dominate their competitors on a ca…
For normal software it would be as you say, but LLM progress is so ridiculously fast that things go from "bleeding edge" to "eh, you'll do" in about that timeframe, and "eh, you'll do" to "why even bother with this old rubbish?" in the same again.
Or, from a different perspective, we can expect some new frontier model from Anthropic in a week or two, and from OpenAI in a month or so.