Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Discovering Cryptographic Weaknesses with Claude

anthropic.com

Post reply on HN
Print view View original

81–90 of 199 posts

Re: Discovering Cryptographic Weaknesses with Claude

#81

Post by 8note »

staticshock wrote: post #47

When high quality effort is applied to a tool, such as AES or the linux kernel, we intuit that it "hardens" the tool. That is, it makes the tool more correct, more resilient, less assailable, etc. Similarly, when effort is applied to an open problem, such as the Riemann hypothesis or P v NP, without progress, it "hardens" the problem: it makes the problem feel more daunting to whoever takes a stab at it next. Andrew…

i think id almost worry more that ai can solve problems in latent space that it cant translate back to tokens because decoding ruins it, and that we wont be able to come up with concepts that we can map to properly decode those solutions in a way people understand

Re: Discovering Cryptographic Weaknesses with Claude

#82

Post by alwa »

connorboyle wrote: post #78
_dwt wrote: post #22

I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good". Friends, look at the prompts that Anthropic's own people are putting into the machine: > A few hours after the first message, we found that Claude was st…

Been skeptical for quite some time that "using AI" is a real skill/genuine field of expertise. I think I need to start sharing that skepticism more loudly and publicly! (I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization,…

I think knowing what you want (and being able to articulate it with adequate specificity, and detecting when processes drift away from it) is a real skill/field of expertise. I just don't think it's specific to interfacing with "AI."

Re: Discovering Cryptographic Weaknesses with Claude

#83

Post by mw888 »

_dwt wrote: post #22

I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good". Friends, look at the prompts that Anthropic's own people are putting into the machine: > A few hours after the first message, we found that Claude was st…

In toying with VS Code or other 'agent harnesses' I'm always annoyed to find very long verbose system prompts "You are CoPilot. You are an expert. You can do anything you put your mind to..." and on and on.

The token cost is amortized for longer conversations, but I find it bothersome that there's all this implicit instruction I didn't write or am now obligated to understand.

I make a custom agent prompt with "Defer to the user." and little else.

Re: Discovering Cryptographic Weaknesses with Claude

#84

Post by daishi55 »

connorboyle wrote: post #78
_dwt wrote: post #22

I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good". Friends, look at the prompts that Anthropic's own people are putting into the machine: > A few hours after the first message, we found that Claude was st…

Been skeptical for quite some time that "using AI" is a real skill/genuine field of expertise. I think I need to start sharing that skepticism more loudly and publicly! (I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization,…

In this case they didn’t know what they were looking for. They just wanted some new, non-trivial exploit.

For the vast majority of corporate usage of AI for SWE, you have a much better idea of what you want, or what the problem is, etc etc. And communicating that to the model effectively is absolutely a skill. I see colleagues every day who very much do not have that skill.

Re: Discovering Cryptographic Weaknesses with Claude

#85

Post by IsTom »

ameliaquining wrote: post #39
vrighter wrote: post #35

Earlier quoted context omitted.

good thing quantum computers that can factor numbers have never been built. No number was ever really factored without cheating, the actual shor's algorithm has never been implemented. And we're not really any closer to

That last sentence is not true; we have gotten much closer to building a quantum computer that can run Shor's algorithm. Organizations like Google and Cloudflare have declared a 2029 deadline to completely stop depending on the security of pre-quantum algorithms; hitting that deadline is going to cost a lot of engineering resources, but they're paying that cost because they think there's too great a chance that natio…

Have they factored 21 yet? That metric hasn't changed for some time.

Re: Discovering Cryptographic Weaknesses with Claude

#86

Post by nozzlegear »

heaney-555 wrote: post #42
ecshafer wrote: post #17

Earlier quoted context omitted.

So $1-10k in Chinese model time, thus why we must ban them.

If a Chinese model can do it for $1-10K, then why hasn't one? Why have all the mathematical (and now cryptographic) breakthroughs come from OpenAI and Anthropic? Is it possibly because the Chinese models are so benchmaxxed they can't make novel discoveries?

Perhaps since the Chinese companies don't have the economy of an entire global superpower riding on them, they can't afford to light $100k+ on fire doing random shit with the hope it'll turn out a research paper.

Re: Discovering Cryptographic Weaknesses with Claude

#87

Post by niccl »

daishi55 wrote: post #84
connorboyle wrote: post #78

Earlier quoted context omitted.

Been skeptical for quite some time that "using AI" is a real skill/genuine field of expertise. I think I need to start sharing that skepticism more loudly and publicly! (I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization,…

In this case they didn’t know what they were looking for. They just wanted some new, non-trivial exploit. For the vast majority of corporate usage of AI for SWE, you have a much better idea of what you want, or what the problem is, etc etc. And communicating that to the model effectively is absolutely a skill. I see colleagues every day who very much do not have that skill.

I agree with this in general but I think there's a nuance. 'dumb' prompts can get the result, but require more back-and-forth with the model and more token usage when compared to a crafted prompt

Re: Discovering Cryptographic Weaknesses with Claude

#88

Post by xmcp123 »

Diogenesian wrote: post #27

This seems like a bit of an overstatement: Despite HAWK having survived two rounds of expert human review over a period of two years, Mythos was able to improve the best-known attack on it in just 60 hours of work—effectively cutting its key strength in half. since, later: Mythos’s attack works by finding a specific, previously unexploited symmetry called a nontrivial automorphism in the lattice used by HAWK. Prior w…

I think it would actually be very valid to downplay mythos’s contribution based on what you said.

It’s the difference between having an original thought or the ability to extrapolate one based on data vs the ability to ingest someone else’s thought and validate/expand on it.

That is a huge difference.

Re: Discovering Cryptographic Weaknesses with Claude

#89

Post by _zoltan_ »

a-dub wrote: post #2

> The multi-agent workflow led to interesting dynamics. For example, the key idea in producing this attack was discovered by a pair of workers working together. Both started investigating the idea; the first worker prematurely rejected the idea as infeasible, but the second found a way to fully exploit it. The pair kept exchanging messages, and eventually both agreed they had found an effective attack. this is pretty…

but in ultracode subagents don't communicate back and forth, do they? so this must have been an other harness than Claude code?

Re: Discovering Cryptographic Weaknesses with Claude

#90

Post by daishi55 »

niccl wrote: post #87
daishi55 wrote: post #84

Earlier quoted context omitted.

In this case they didn’t know what they were looking for. They just wanted some new, non-trivial exploit. For the vast majority of corporate usage of AI for SWE, you have a much better idea of what you want, or what the problem is, etc etc. And communicating that to the model effectively is absolutely a skill. I see colleagues every day who very much do not have that skill.

I agree with this in general but I think there's a nuance. 'dumb' prompts can get the result, but require more back-and-forth with the model and more token usage when compared to a crafted prompt

For sure. And honestly the real skill is knowing when you can be lazy and when you will get a good ROI on a good prompt. Sorry I sounded a little like Claude there but that’s what I meant to say.
Post reply on HN