Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Project Glasswing: what Mythos showed us

blog.cloudflare.com

Post reply on HN
Print view View original

141–150 of 152 posts

Re: Project Glasswing: what Mythos showed us

#141

Post by rithdmc »

sandeepkd wrote: post #48
wslh wrote: post #40

Earlier quoted context omitted.

In the last few days I was recommending to read the insights from XBOW [1], it's a competitor but it adds more information to the discussion. [1] https://xbow.com/blog/mythos-offensive-security-xbow-evaluat...

Thanks for sharing. Its definitely more concrete. Some of the things that I was hoping to find were, the number of false positives, the times it takes to identify the false positives from real ones, the taxation on human mind to perform this exercise. Did anyone manually verified the exploits which were identified by the LLM or were they assumed correct based on the explanation. I do understand that the target audien…

I don't understand how XBOW measured the false negative rates.

Re: Project Glasswing: what Mythos showed us

#142

Post by alsetmusic »

__natty__ wrote: post #51
roxolotl wrote: post #41

What does this mean? > It's a different kind of tool doing a different kind of work, and that makes a clean apples-to-apples comparison to earlier models difficult. They claim it’s a different kind of tool and then describe using it the same way you’d use any other model. This really felt way worse than the average Cloudflare blog and really just rehashed the Mythos announcement which had already called out the key p…

Sounds different because it’s hidden advertisement not a regular blog post

> Sounds different because it’s hidden advertisement not a regular blog post

Yep. Cloudflare has lost my respect over the last six months.

The posts about pro-AI initiatives and APIs for AI and then laying off a lot of people was pretty impressive for how to do the wrong thing.

Re: Project Glasswing: what Mythos showed us

#144

Post by Archit3ch »

> Programming language - C and C++ give you direct memory control and, with it, bug classes - buffer overflows, out-of-bounds reads and writes - that memory-safe languages like Rust eliminate at compile time. We saw consistently more false positives from projects written in memory-unsafe languages.

Re-write your Rust into C++ to drown the attacker in false positives? ;)

Re: Project Glasswing: what Mythos showed us

#145

Post by alexjplant »

ajmurmann wrote: post #126
alexjplant wrote: post #55

Earlier quoted context omitted.

Let's double-click on that. It's important to keep top of mind that using disruptive words and patterns in conversation isn't always driven by LLMs — reasoning from first principles tells us that problematic usages like this existed beforehand. One of my load-bearing career learnings is that people used this shape of language as a shibboleth long before game-changing tools like ChatGPT started slopping so much of wha…

I don't think it's performative or about vibes. Everyone subconsciously adopts phrases and in general ways of talking from people around them. May it be from friends, neighbors or coworkers.

Not incompatible with my satirical post (I wrote "performant," a notorious tech neologism, not performative). Whether subconsciously or not people 100000% use language to communicate and determine others' social tribe membership.

Re: Project Glasswing: what Mythos showed us

#146

Post by sn0rl27 »

meander_water wrote: post #98
roxolotl wrote: post #41

What does this mean? > It's a different kind of tool doing a different kind of work, and that makes a clean apples-to-apples comparison to earlier models difficult. They claim it’s a different kind of tool and then describe using it the same way you’d use any other model. This really felt way worse than the average Cloudflare blog and really just rehashed the Mythos announcement which had already called out the key p…

> the model has its own emergent guardrails that sometimes cause it to push back on legitimate security research requests. But as we found, these organic refusals aren’t consistent - the same task, framed differently or presented in a different context, could produce completely different outcomes as illustrated in the examples below. This was new. I'm surprised that a model specifically designed for security research…

The model wasn’t created specifically for security research. It’s a general model that just happens to be dangerously good at security research (according to Anthropic)

Re: Project Glasswing: what Mythos showed us

#147

Post by Avicebron »

ctoth wrote: post #44
Avicebron wrote: post #29

Earlier quoted context omitted.

That's a scary thought, llm's training on llm output. People trained by default of ubiquity to think and read llm output produce their own llm-esque writing. Seems stifling. We'll need someway to reward human creativity and out-of-bounds thinking before our greatest corpus of human intellect is a bounded by whenever and whatever was trained on.

So is it that humans are inherently creative, machines could never do what we do? Or is it that humans will only replicate our training data, and so we have to ensure that machines don't bound our training data? Or are you going meta and gently pointing out the absurdity? (I hope it's this one!)

I think I have an answer. Human's don't have "training data" in the same way we think of LLMs, yes you can walk outside your house and quantify every electromagnetic pulse, random pertubation etc and then "train on it". But that isn't how people process information. We have the ability to process our entire "existence" if that makes sense, which means the density is much higher.

The LLM is bounded by it's training data, and relying on it means we are as well.

Re: Project Glasswing: what Mythos showed us

#148

Post by tskj »

krupan wrote: post #91
tskj wrote: post #82

Earlier quoted context omitted.

Yeah why not? That's how I work. If I don't review my work, it's way worse than if I do review it and revise and iterate. I don't see why AI should be different: in fact it very clearly seems to be the case that is isn't.

I mean, I was sold something different. Something super human, vastly more intelligent, world changing. The reality is not that. Am I allowed to be disappointed and discouraged?

Because you can have it review itself and iterate on its own work before showing you. If you insist on reviewing its one-shot output you'll be disappointed, but if you consider its internal work private and only consider its final output, it's different.

Also we're still in the middle of the transformation, clearly the AI we'll have in 5 years will be radically different and better (by some definition of better) than what we see today. It's kind of weird that you'd be disappointed that the world will only be totally transformed in ten years, and not five.

Re: Project Glasswing: what Mythos showed us

#149

Post by tskj »

kllrnohj wrote: post #106
tskj wrote: post #80

Earlier quoted context omitted.

I don't understand this mindset, why is it people on here think humans have some kind of magical ability machines don't or can't? Five years ago I would never have predicted this kind of human chauvinism here. It's some kind of weird romanticism almost.

Because right now humans do have a magical ability machines don't. LLMs are a fuzzy reflection of what they've seen hundreds of times already, they don't have originality or intelligence (yet). As a much more immediate practical matter, LLMs trained on LLM output makes them worse overall, they degrade from doing that. So the more LLM-prodoced content fills the web, the less useful it is as a data source for future LL…

Saying they don't posses any level of intelligence is wild.
Post reply on HN