Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts about Anthropic's new cryptanalysis results

blog.cryptographyengineering.com

Post reply on HN
Print view View original

91–100 of 124 posts

Re: Some thoughts about Anthropic's new cryptanalysis results

#91

Post by iansmith_hn »

Does anybody have a filter for cryptography posts of the form:

AES IS BROKEN: Making giant assumption XYZ and requiring a less capable AES in ABC way, we've reduce the amount of operations needed to break AES from 10^X to 10^X-1!

These are tedious for people who are interested in cryptography but are not researchers in the field. (For the researchers, this type of thing may be useful.). The fact that AI is now "generating crypto results", suggests that soon we will soon have crypto-post-slop as clickbait...

Re: Some thoughts about Anthropic's new cryptanalysis results

#92

Post by dandaka »

jiggawatts wrote: post #54
simonw wrote: post #6

This is good: > If you’re under the impression that these models are “glorified autocomplete” or that progress is slowing down, I need to urge you: stop thinking that . The models are very intelligent and capable, they are getting better at a fast clip. I can cite measurable and impressive progress over just the past five months on specific types of problem I’ve asked them to look at. [...] > On the other hand : if y…

My mental model is this: There is a vast ocean of human knowledge, far beyond the capacity of any human brain, even within specialised fields. Books helped "plug the gaps" in our knowledge, increasing the scope that a single human mind can encompass. Web search engines did the same thing, but more and faster. LLMs are like search engines on steroids, essentially a research librarian that operates at 1,000x human spee…

Great summarization of current capabilities of models, thank you

Re: Some thoughts about Anthropic's new cryptanalysis results

#93

Post by amenhotep »

CamperBob2 wrote: post #63
simonw wrote: post #22

Earlier quoted context omitted.

I think understanding how this stuff works is really important. For technical people it gives them a useful starting point for understanding it all. For less technical people it's crucial to help them understand that it's not some weird new magical science-fiction AI - it's still computer programs that turn text into numbers and do stuff with the numbers and turn those back into text. It's harder to believe something…

It's harder to believe something is conscious or threatening to achieve word domination once you understand that it's a machine that statistically figures out which word should come next. The problem is, these models challenge our definition of "consciousness." Or at least they point out how hopelessly-inadequate our thinking on the subject is. Some people really, really don't like having their personal definition of…

LLMs self-evidently have no experience when not inferencing, seems like the most consequential difference to me. If we are next token predictors then we are next token predictors that are inferencing at every waking moment and arguably much of our sleeping moments too; when we train an LLM that can inference as much as we do and remain coherent then I'll be more worried about whether it might have experience.

Re: Some thoughts about Anthropic's new cryptanalysis results

#94

Post by irishcoffee »

tptacek wrote: post #84
sph87 wrote: post #65

Earlier quoted context omitted.

> AGI is already here I feel like there has been a ton of noise about this, but frankly, no one has actually defined what AGI means. I feel like the goal post is constantly shifting. Take for example Humanity's Last Exam. It is so broad and complex that while an individual in a specific field might be able to answer their specific area of questions, they certainly would not be able to achieve >50% on the total questi…

Subtextually, when people discuss AGI as a threshold, they're talking about superintelligence, which I'd roughly define as reasoning (not just applying algorithmic search) at the speed with which computers do lookups and computations, with large-scale data set contexts. Right now, models are doing well-defined knowledge work tasks, applying relatively well-worn patterns (but with the thoroughness of a computer, which…

Super intelligence is reasoning quickly? That’s what we’ve reduced it to?

Don’t tell Mozart.

Re: Some thoughts about Anthropic's new cryptanalysis results

#95

Post by brookst »

annzabelle wrote: post #67
sph87 wrote: post #65

Earlier quoted context omitted.

> AGI is already here I feel like there has been a ton of noise about this, but frankly, no one has actually defined what AGI means. I feel like the goal post is constantly shifting. Take for example Humanity's Last Exam. It is so broad and complex that while an individual in a specific field might be able to answer their specific area of questions, they certainly would not be able to achieve >50% on the total questi…

There's a specific singularity theory of AI that is very popular. Eliezer Yudkowsky helped popularize it among the Bay Area "rationalist" community, and it has this idea that AGI necessarily implies a self improving system that will quickly become a paperclip maximizer or other such dystopian or utopian world changing intelligence. By that singularity definition, we're probably nowhere near AGI, but if we define it a…

The quotes around rationalist should be mandatory.

Re: Some thoughts about Anthropic's new cryptanalysis results

#96

Post by gwd »

dgellow wrote: post #14
dboreham wrote: post #12

Earlier quoted context omitted.

I'm also getting irritated with the “glorified autocomplete” comments. Since nobody can post such comments and also use the tools I'm using, I'm wondering if the phenomenon is due to people only having experience with the free version of whatever it is they're trying to use?

The “glorified autocomplete” framing isn’t to take literally. It’s a way to remove the mystic and whole anthropomorphization of AI. It’s saying they aren’t sentient or entities we are interacting with, even if that’s how the output presents itself. Instead they are “just” stochastic models

There was an interview I watched with Cory Doctorow just a few weeks ago, where someone asked him, "What kind of evidence could be presented to you to make you believe that LLMs were now ready to replace people at jobs entirely?" And he basically responded that he didn't think it was possible they ever would, because they're not really thinking, they just "predict the next token". This in spite of him actually having first-hand experience with people getting loads of value out of Claude Code and GPT Codex.

Re: Some thoughts about Anthropic's new cryptanalysis results

#97

Post by brookst »

kijin wrote: post #74
annzabelle wrote: post #67

Earlier quoted context omitted.

There's a specific singularity theory of AI that is very popular. Eliezer Yudkowsky helped popularize it among the Bay Area "rationalist" community, and it has this idea that AGI necessarily implies a self improving system that will quickly become a paperclip maximizer or other such dystopian or utopian world changing intelligence. By that singularity definition, we're probably nowhere near AGI, but if we define it a…

We'll probably be able to tell if and when Yudkowsky's AGI arrives. Once AI can improve itself, frontier labs will no longer need human developers. So we might see a massive layoff of top talent and a dramatic increase in product quality at the same time. This usually doesn't happen in human businesses. It is also very much against the interest of anyone who is already at the top of the pay table at those labs.

AI is already improving itself. Most / all of the coding harnesses are AI-written.

And yet… there are still people telling AI how to improve itself.

IMO there will always be a level of abstraction at which AI needs guidance. Perhaps ASI means it decides everything on its own, but I don’t think so. Genius humans often excel at the how but not the why, or even the what. So far there’s no indication that AI is different.

Re: Some thoughts about Anthropic's new cryptanalysis results

#98

Post by gwd »

simonw wrote: post #4

> both outputs of Claude Mythos, their (still) unreleased advanced model That sentence gives the impression that Mythos might be released in the future. That's clearly not going to happen - it's already "released" in as much as selected, trusted partners can access it, and the rest of us get it in the form of Fable - which is Mythos but with filters that downgrade you if you try to use it for anything even remotely r…

Got downgraded from Fable to Opus after asking about the Great Oxidation Event [1], presumably because it started thinking about cyanobacteria, which made the monitor afraid I was trying to make a biological weapon or something.

[1] https://en.wikipedia.org/wiki/Great_Oxidation_Event

Re: Some thoughts about Anthropic's new cryptanalysis results

#99

Post by brookst »

gwd wrote: post #96
dgellow wrote: post #14

Earlier quoted context omitted.

The “glorified autocomplete” framing isn’t to take literally. It’s a way to remove the mystic and whole anthropomorphization of AI. It’s saying they aren’t sentient or entities we are interacting with, even if that’s how the output presents itself. Instead they are “just” stochastic models

There was an interview I watched with Cory Doctorow just a few weeks ago, where someone asked him, "What kind of evidence could be presented to you to make you believe that LLMs were now ready to replace people at jobs entirely?" And he basically responded that he didn't think it was possible they ever would, because they're not really thinking, they just "predict the next token". This in spite of him actually having…

Doctorow is one of those people who makes money from what he believes, so it’s very unlikely he’ll ever change his beliefs.

Re: Some thoughts about Anthropic's new cryptanalysis results

#100

Post by brookst »

dgellow wrote: post #14
dboreham wrote: post #12

Earlier quoted context omitted.

I'm also getting irritated with the “glorified autocomplete” comments. Since nobody can post such comments and also use the tools I'm using, I'm wondering if the phenomenon is due to people only having experience with the free version of whatever it is they're trying to use?

The “glorified autocomplete” framing isn’t to take literally. It’s a way to remove the mystic and whole anthropomorphization of AI. It’s saying they aren’t sentient or entities we are interacting with, even if that’s how the output presents itself. Instead they are “just” stochastic models

It’s such willful blindness though. Why deny that there is a huge gulf between autocomplete and sentience? It’s like saying cars are glorified bicycles because they’re sure not airplanes.
Post reply on HN