Live data from Hacker News

Discovering Cryptographic Weaknesses with Claude

anthropic.com

161–170 of 199 posts

Re: Discovering Cryptographic Weaknesses with Claude

#161
post #22

I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good". Friends, look at the prompts that Anthropic's own people are putting into the machine: > A few hours after the first message, we found that Claude was st…

That's reassuring, it's exactly how I've always done LLM. It's fine, as long as the gist of what you mean is there you can be real lazy when typing. And I've always been on the cheapest of plans.

Re: Discovering Cryptographic Weaknesses with Claude

#162
post #22

I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good". Friends, look at the prompts that Anthropic's own people are putting into the machine: > A few hours after the first message, we found that Claude was st…

Been skeptical for quite some time that "using AI" is a real skill/genuine field of expertise. I think I need to start sharing that skepticism more loudly and publicly! (I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization,…

> Been skeptical for quite some time that "using AI" is a real skill/genuine field of expertise.

I feel using AI (effectively) is not too far from the skillset of programming. It is still a machine following instructions (just, maddenigly non-deterministic, but still close enough), so the same insticts of breaking down work into clearly defined sequences that make a good programmer also make a good AI jockey.

Re: Discovering Cryptographic Weaknesses with Claude

#163
post #114

Earlier quoted context omitted.

Indeed. It sounds more like management (Minus the 'people skills' since you don't technically need Claude to like you... yet).

I'm curious if there has been any study on the output quality of LLMs after actively antagonizing them. It's easy to forget that at the most basic level these are just auto-complete machines, and humans tend to not respond so well after antagonized. It stands to reason that the same should follow for an auto-complete system trained on human interaction.

Intuitively I'd say lashing out on an LLM overconstraints it and so worsens the output, therefore the skill to stay calm and neutral is useful for guiding a machine as well. Would be useful to have data on that.

Re: Discovering Cryptographic Weaknesses with Claude

#164
post #82

Earlier quoted context omitted.

Been skeptical for quite some time that "using AI" is a real skill/genuine field of expertise. I think I need to start sharing that skepticism more loudly and publicly! (I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization,…

I think knowing what you want (and being able to articulate it with adequate specificity, and detecting when processes drift away from it) is a real skill/field of expertise. I just don't think it's specific to interfacing with "AI."

It is basically the same skillset than asking a good stackoverflow question. Provide enough context and state your problem clearly

Re: Discovering Cryptographic Weaknesses with Claude

#165
post #22

I find that some of my friends and acquaintances have gotten obsessed with prompting style, "prompt engineering", which skills to use, which skills to build, "context engineering", and a billion other variations on "how to write smart things so the model does good". Friends, look at the prompts that Anthropic's own people are putting into the machine: > A few hours after the first message, we found that Claude was st…

Keep in mind for Anthropic the goal for them isn't necessarily to solve a problem but to demonstrate/measure the innate capabilities of their model. Many results would be a lot less interesting/impressive for an LLM to accomplish if researchers smuggled in key premises or ideas within their prompt.

Re: Discovering Cryptographic Weaknesses with Claude

#166

'Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim, and to prepare the research paper itself, which we are releasing along with this blog post' So model outputs something, that can be completely bogus, and a lot of people spend a lot of hours checking if it's worth anything(not for the sake of science, but for the sake of publishing and marke…

> (not for the sake of science, but for the sake of publishing and marketing) I would argue that it is science. > And then even more people need to spend even more hours to understand that paper? Well, yes, it's kinda the point of reading any paper. > Can I apply for a position Sure you can, https://www.anthropic.com/careers/jobs

Re: Discovering Cryptographic Weaknesses with Claude

#167

When high quality effort is applied to a tool, such as AES or the linux kernel, we intuit that it "hardens" the tool. That is, it makes the tool more correct, more resilient, less assailable, etc. Similarly, when effort is applied to an open problem, such as the Riemann hypothesis or P v NP, without progress, it "hardens" the problem: it makes the problem feel more daunting to whoever takes a stab at it next. Andrew…

AI is nowhere near the intelligence of a very educated person that has innate talent for problem solving. It does solve the problem of applying human intelligence on problems that truly need it. AI is also a great tool to see if there's something simple that we've missed or just haven't even attempted due to wrong assumptions.

In this case it's not actually that big a result, it's an incremental improvement on a series of previous results. Cryptographer Orr Dunkelman describes it better than I ever could, he's one of the people who produced one of the previous results:

  The main result in this paper is improving the Derbez, Foque, Jean attack from EUROCRYPT 2013, which is an improvement of our attack from CRYPTO 2010, which is an improvement of the Demirci-Selcuk attack, which is the improvement of the Gilbert-Minier collision attack against 7-round attack [...]

  To save everybody's time, the [DFJ13] attack is on 7-round AES. The new result is also on 7-round AES, "eroding" the security margin of 7-round AES by about 8 bits of security [...] While this is the first improvement in attacking 7-round AES in the last decade, if you were not worried by the series of papers that reduced the security of 5-round AES from 2^32 to 2^16, or the somewhat improved attacks on 6-round AES, then you should not really worry now to start a procedure for changing 10-round AES (for 128-bit key) for something else, when there are no attacks on 8-round AES-128.
So someone threw a clanker at a series of previous results and told it to find improvements. Since it's ingested every piece of crypto research ever and can draw on all of them instantly, it managed to tweak the previous work a bit to improve the attack slightly... on a version of AES deliberately weakened to make attacks easier, a standard procedure for any iterated crypto algorithm where you see how many rounds you can get into it before your attack stalls. So it's a bit like saying you knocked out Mike Tyson's brother's cousin's uncle's sister's nephew in four rounds instead of five.

Re: Discovering Cryptographic Weaknesses with Claude

#168
post #72

Earlier quoted context omitted.

Never once had to do that, fwiw.

You'll get better code if you do

i use it primarily to enforce my personal conventions that are not super commonly followed in the wild. structure of comments at the top of .h , structure of bibliography in .tex, etc etc things that i would otherwise have to repeat periodically.

Re: Discovering Cryptographic Weaknesses with Claude

#169

Earlier quoted context omitted.

If a Chinese model can do it for $1-10K, then why hasn't one? Why have all the mathematical (and now cryptographic) breakthroughs come from OpenAI and Anthropic? Is it possibly because the Chinese models are so benchmaxxed they can't make novel discoveries?

“I haven’t seen X, therefore it doesn’t exist.” Absence of evidence is not an evidence for absence of something.

I haven't seen leprechauns, at least not without the aid of xiao ren ren mushrooms, but I'm still pretty sure they don't exist.

Re: Discovering Cryptographic Weaknesses with Claude

#170

Earlier quoted context omitted.

Having domain expertise isn't specific to using AI. Not being able to tell when it's hallucinating has led to some very adverse outcomes.

Including for about 1809 lawyers https://damiencharlotin.com/hallucinations

Interesting
Post reply on HN