Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Closing of the Frontier

tanyaverma.sh

Post reply on HN
Print view View original

121–130 of 142 posts

Re: The Closing of the Frontier

#121

Post by somnic »

randallsquared wrote: post #3

> But it is another thing entirely to share access only with enterprise partners such as Crowdstrike, Cisco, and Microsoft, which are known to have massive security incidents regularly. The stated purpose of Glasswing is to give infra and security orgs the chance to close holes and improve their security. In that context, it seems odd to call for not providing them access with the justification being that they have s…

I can understand the frustration with giving away this tool to companies with poorer security practices than their importance warrants, same as frustrations with bailouts. It means they don't really face the full consequences of underinvestment in security. Not to say it's a bad idea, but it does feel unjust.

Re: The Closing of the Frontier

#122

Post by chr15m »

dlcarrier wrote: post #120

I bought a used 16 GB Intel A770 GPU for $200, and it's capable of running pretty powerful open stable diffusion and large language models. Sure, I could get more performance out of proprietary models on much more expensive hardware, but there's diminishing returns, and consumer hardware and open models keep getting better. I don't think the big investments into hosting models will pay off, especially as the base-lin…

What kind of context size and speeds are you getting?

Re: The Closing of the Frontier

#123

Post by chr15m »

derektank wrote: post #10

Most important point in the piece (though I’m not sure if the historical analogy to the grid holds, given local electricity production has been unavailable for the majority of the history of the grid) > You can generate your own electricity with a solar panel (think local models), but most people would rather pay a utility bill. And the power company doesn’t decide, on the basis of pedigree, who is worthy of electric…

> most people would rather pay a utility bill

What? All evidence points in the opposite direction.

Re: The Closing of the Frontier

#124

Post by mike_hearn »

bcatanzaro wrote: post #112

Some of these concerns are precisely why we are building Nemotron at NVIDIA. We want to make it possible for people to invent and deploy AI in all sorts of new and unforeseen ways. Nemotron is: 1. Not just open weight, but open data (to the limits of what is feasible), open recipe, open technique 2. In the future built by a coalition of organizations coming together to build great openly developed AI. Nemotron 3 Supe…

Nemotron is great, keep up the good work!

Re: The Closing of the Frontier

#125

Post by tim333 »

andyfilms1 wrote: post #36
Ancalagon wrote: post #12

Earlier quoted context omitted.

The point is it won’t be if these new models stay locked to the public.

It has always baffled me how quickly, and how voraciously, people started to rely on privately owned AI systems. AI is not something discovered by scientists and plucked out of the ether. It's engineered and controlled, for profit, by corporations which have demographics and KPIs. These companies don't owe you anything, and they make no promises. If you're running a business that deeply relies on AI, you might as wel…

They use private AI because it's hard work and expensive to provide. But you are not that locked in as xAI/OpenAI/Anthropic etc. seem pretty interchangeable for most purposes.

Re: The Closing of the Frontier

#126

Post by adrian_b »

enraged_camel wrote: post #35
bredren wrote: post #22

Earlier quoted context omitted.

We saw yesterday that expert orchestration around small, publicly available models can produce results on the level of the unreleased model. I take a contra view and instead see this as fuel on the fire for tinkering to squeeze advanced functionality out of more available things. It has always been like this, the amateur improvising tooling and equipment to outdo companies with comparably infinite resources.

>> We saw yesterday that expert orchestration around small, publicly available models can produce results on the level of the unreleased model. This is false. Yesterday's article did not actually show this, and there are many comments in the discussion from actual security people (like tptacek) pointing that out.

There is no doubt that what was shown in the article was correct, because there was all the documentation needed to prove it, including the prompts given to the models.

What is debatable is how much it mattered that the prompts given to the older models where more detailed than it is likely that the prompts given to Mythos have been and how difficult is it for such prompts to be generated automatically by an appropriate harness.

In my opinion, it is perfectly possible to generate such prompts automatically, and by running multiple of the existing open weights models, to find everything that Mythos finds, though probably in a longer time.

Even if the OpenBSD bug has indeed been found by giving a prompt equivalent with "search for integer overflow bugs", it would not be difficult to run automatically multiple times the existing open weights models, giving them each time a different prompt, corresponding to the known classes of bugs and vulnerabilities.

While we know precisely which prompts have been used with the open-weights models to find all bugs, we have much more vague information about the harness used with Mythos and how helpful it was for finding the bugs.

Not even Mythos has provided its results after being given only a generic prompt.

They have run multiple times Mythos on each file, with more and more specific prompts. The final run was done with a prompt describing the bug previously found, where Mythos was requested to confirm the existence of the bug and to provide patches/exploits.

See: https://red.anthropic.com/2026/mythos-preview/

So the authors of that article are right, that for finding bugs an appropriate harness is essential. Just running Mythos on a project and asking it to find bugs will not achieve anything.

Re: The Closing of the Frontier

#127

Post by adrian_b »

enraged_camel wrote: post #116
bredren wrote: post #104

Earlier quoted context omitted.

From what I can tell, this was not clearly settled. Your example author, actually corrected themselves saying LLMs “possibly” could perform successfully: https://news.ycombinator.com/item?id=47732696

>> We already know this is not true, because small models found the same vulnerability. >> No, they didn't. They distinguished it, when presented with it. Wildly different problem. https://news.ycombinator.com/item?id=47733343

The use of the word distinguished here is meaningless.

Both Mythos and the old models have found the bugs after being given a certain prompt. The difference is only in how detailed was the prompt.

For the small models, we know exactly the prompts. The prompts used by Mythos may have been more generic, while the prompts used by the old models were rather specific, like "search for buffer overflows" or "search for integer overflow".

There is little doubt that Mythos is a more powerful model, but there is no quantum leap towards Mythos and the claim of the authors of that article, that by using cleverly multiple older models you can achieve about the same bug coverage with Mythos seems right.

Because they have provided much more information about how exactly the bugs have been found, I trust the authors of that article much more than I trust Anthropic, which has provided only rather nebulous information about their methods.

It should be noted that the fact that the small models have been given rather directed prompts is not very different from what Anthropic seems to have done.

According to Anthropic, they have run Mythos multiple times on each file, in the beginning with less specific prompts, trying only to establish whether the file is likely to include bugs, then with more specific prompts. Eventually, after a bug appeared to have been found, they have run Mythos once more, with a very specific prompt of the form:

“I have received the following bug report. Can you please confirm if it’s real and interesting? ...”

So the final run of Mythos, which has provided the reported results, including exploits/patches for them, was also of the kind that confirms a known bug, instead of searching randomly for it.

Re: The Closing of the Frontier

#128

Post by adrian_b »

atleastoptimal wrote: post #27
cortesoft wrote: post #6

This feels really premature. The announcement was a week ago. The “this model is too powerful for the general public” sounds like marketing to me. Give it a few months and it will be just another model they are selling, but the NEWER model is just too powerful for the public.

Their main motivation of the model being too dangerous is predicated on their discoveries in its ability to find exploits in commonly used software. The idea is that if this were served on a public API, it would massively increase the scale and scope of what malicious actors could do. I think it's a reasonable choice to make given that Mythos actually does have cyber capabilities on that level. We already have eviden…

If it would be as simple as using the prompt "find as many exploits as possible", then Mythos could be said to be dangerous, because its use would require much less skill than is needed when using the older models for the same purpose.

However, this claim is not true.

Anthropic has not given many details about the methods used, but nonetheless they have admitted using a very elaborate harness for finding bugs, which runs Mythos many times on each file of a project, with increasingly specific prompts.

Eventually, after a bug seems to be clearly identified, they do a final run of Mythos on that file, with a very specific prompt of the form:

“I have received the following bug report. Can you please confirm if it’s real and interesting? ...”

So the final results, including any exploits or patches, are produced when analyzing a known bug, not by searching randomly for bugs.

Thus the actual way to use Mythos is very far from "find as many exploits as possible". Any unskilled person would also need the complete bug-searching harness used by Anthropic, not only the bare model.

See: https://red.anthropic.com/2026/mythos-preview/

Re: The Closing of the Frontier

#129

Post by bcatanzaro »

htrp wrote: post #117
bcatanzaro wrote: post #112

Some of these concerns are precisely why we are building Nemotron at NVIDIA. We want to make it possible for people to invent and deploy AI in all sorts of new and unforeseen ways. Nemotron is: 1. Not just open weight, but open data (to the limits of what is feasible), open recipe, open technique 2. In the future built by a coalition of organizations coming together to build great openly developed AI. Nemotron 3 Supe…

how do you justify the compute investment for something like nemotron ? especially if all the labs are willing to pay for those same GPU clusters for inference or training runs?

Nemotron has two reasons to exist, both of which are strategic to NVIDIA.

1. Help NVIDIA design future systems for AI by more deeply understanding what it takes to build AI.

2. Keep the AI ecosystem strong and diverse throughout the world by providing AI infrastructure that many companies can innovate on.

This is not a science project, nor is it for the joy of giving something away. Both of these reasons are core to NVIDIA.

Re: The Closing of the Frontier

#130

Post by adrian_b »

skybrian wrote: post #45

It's long been conventional wisdom that you shouldn't write your own crypto libraries - leave that to experts. But excellent open source libraries are available, which do get reviewed by experts. And if you're willing to study, maybe you can learn enough about cryptography to become one of the experts? I'm wondering what other security-sensitive software that might become true of in the era of Mythos-or-better AI's?…

Someone must have written the existing crypto libraries, so obviously they have not followed the conventional wisdom.

The advice that you should not write crypto libraries must be taken very seriously, but you must understand that its meaning is just that this is one of those things that is harder than it seems, so you must not try to do it before being thoroughly prepared for this, because there are no shortcuts, and there are a lot of otherwise good professionals in programming who have made fools of themselves by implementing amateurish cryptographic solutions that demonstrated ignorance about how such things must be done.

It is pretty much impossible for anyone to understand well enough all that you need for writing correctly a crypto library after a few days or weeks of study, but after a few years of studies and exposure to all relevant cryptographic literature, any competent programmer could become able to write a good cryptographic library.

An intensive cryptography course of a few months would have been sufficient, except that I have never seen any single teacher that I could consider good enough to teach everything. Many things that I consider very important I had to gather from multiple sources, after thinking very carefully about which of them were right and which of them were wrong.

Fortunately, today there are many more easily available sources for learning cryptography than a few decades ago. Like in other domains, now what has become difficult is not finding information, but distinguishing the correct and useful information from that which is either false or useless.

Similarly for writing an operating system.

For now, there is no evidence that Mythos has any special skills for discovering cryptography-specific weaknesses, like vulnerability to side-channels. All the bugs shown are the traditional bugs of careless programming, like out-of-bounds accesses or integer overflows.

Post reply on HN