21–30 of 152 posts
I expressed some concerns along the same lines in the thread about the Mythos evaluation curl did a few days ago, which sounded a lot like the "passing in the repo and telling it go!" type workflow described in this as dramatically less effective.
Disappointed that the post is very slim on details beyond this however. No hard numbers. Not comparatively, not in isolation. Would have arguably been kinda the point.
The real question is whether it was Mythos or Opus that wrote this post. > "Why it matters" It doesn't, it's a corporate blog, they were rarely written in one-author's voice anyway, but it's interesting to see that even large organisations are outsourcing their blogs to LLMs.
I will upgrade the "why it matters" to "and now AI output is part of the training data". A day is coming when the punched-up AI verbiage will be the norm and hard to distinguish unless you're from the previous generation. Sort of in the way that I miss some aspects of Usenet.
That's great and all but how severe were the most severe vulnerabilities found? I imagine they don't want to talk about it, but that's really the most interesting and important bit.
As much as I’d like to share in the skepticism, the very beginning of the article states it very plainly — this is a step function. Lots of people feel that Mythos is a psyops campaign, but I don’t really understand the skepticism. Most of it seems to stem from the general distrust of things that aren’t publicly available. A few Anthropic employees have described Mythos as a general purpose model improvement, but tha…
The real question is whether it was Mythos or Opus that wrote this post. > "Why it matters" It doesn't, it's a corporate blog, they were rarely written in one-author's voice anyway, but it's interesting to see that even large organisations are outsourcing their blogs to LLMs.
It's fascinating seeing people think that if you're snarky enough about something, the substance of that thing actually ceases to be substantive. It's like staring down the barrel of a gun and taking the time to make quips about the type of paper the gun advertisement was printed on.
I can't wait to be told that Cloudflare is now part of "The Mythos FUD" campaign.
I can't wait to be told that Cloudflare is now part of "The Mythos FUD" campaign.
2 things can be true at the same time. I think the curl folks finding it underwhelming is more of a testament to their code being subjected to a lot of tests/attacks/auditing over the past years compared to many other codebases. It's not going to find magically insurmounable exploits on it's own and "pwn teh w0rld". At the same time, there is so much shitty non-memory safe code out there (C/C++ mainly) or logically w…
Static scanners are ok at find a few particular types of issues, and really bad at more abstract issues. Also having rules where you must pass static analysis has to be followed up with actually making sure your code monkeys aren't writing bullshit that confuses the scanner and lets it pass while doing nothing for security (or adding nice logic traps).
Most external security firms looking at code are more useless than a zero with the circle rubbed out. Had a fun example from a while back where the team that wrote the code inserted an intentional security flaw to be sure they were catching anything. Problem is they were giving access to the entire git history so these stood out. The moment they just gave flat code the security teams ability to find flaws disappeared.
LLM models seem to have a pretty good grasp on finding flaws in code like this once you can get the issue to stay in context and execution time. When I hear things like Mythos getting much longer time to work on the problem then at least to me it makes a lot more sense on the number of issues it's picking up.
The real question is whether it was Mythos or Opus that wrote this post. > "Why it matters" It doesn't, it's a corporate blog, they were rarely written in one-author's voice anyway, but it's interesting to see that even large organisations are outsourcing their blogs to LLMs.
Sentence constructions like this definitely scream AI: "That's a reasonable bias for an exploratory tool. It's a ruinous one for a triage queue..." I will upgrade the "why it matters" to "and now AI output is part of the training data". A day is coming when the punched-up AI verbiage will be the norm and hard to distinguish unless you're from the previous generation. Sort of in the way that I miss some aspects of Use…
Seems stifling. We'll need someway to reward human creativity and out-of-bounds thinking before our greatest corpus of human intellect is a bounded by whenever and whatever was trained on.
great, but why don't you share real data on how many security vuln it found ? how many were reals, how many weren't ?
I get that you want to address them or whatever before releasing info but I keep seeing these claims with barely any data and I’m like…how do you expect people to not be skeptical?
I mean hell if you’re a security professional you’re literally paid to be skeptical.