Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts about Anthropic's new cryptanalysis results

blog.cryptographyengineering.com

Post reply on HN
Print view View original

71–80 of 124 posts

Re: Some thoughts about Anthropic's new cryptanalysis results

#71

Post by bawolff »

dgellow wrote: post #14
dboreham wrote: post #12

Earlier quoted context omitted.

I'm also getting irritated with the “glorified autocomplete” comments. Since nobody can post such comments and also use the tools I'm using, I'm wondering if the phenomenon is due to people only having experience with the free version of whatever it is they're trying to use?

The “glorified autocomplete” framing isn’t to take literally. It’s a way to remove the mystic and whole anthropomorphization of AI. It’s saying they aren’t sentient or entities we are interacting with, even if that’s how the output presents itself. Instead they are “just” stochastic models

Which obviously raises the question of if we are just glorified autocomplete (or glorified pattern matching). After all, we evolved from things just trying to recognize the patterns for what will hurt us and what will cause pleasure.

Re: Some thoughts about Anthropic's new cryptanalysis results

#73

Post by red75prime »

argee wrote: post #28
dgellow wrote: post #14

Earlier quoted context omitted.

The “glorified autocomplete” framing isn’t to take literally. It’s a way to remove the mystic and whole anthropomorphization of AI. It’s saying they aren’t sentient or entities we are interacting with, even if that’s how the output presents itself. Instead they are “just” stochastic models

Exactly. They ARE "glorified autocomplete" in an ontological sense. That says nothing about capability or outcome. The people who come out swinging against that characterization usually ignore the whole ontological argument (which is...the entire point) and go after an outcome-based strawman.

Capability or outcome...or internal structure.

What does "glorified autocomplete" say in an ontological sense exactly? Nah. It's just a lazy dismissal.

BTW, autoregressive pretraining (autocomplete) is a part of training.

Re: Some thoughts about Anthropic's new cryptanalysis results

#74

Post by kijin »

annzabelle wrote: post #67
sph87 wrote: post #65

Earlier quoted context omitted.

> AGI is already here I feel like there has been a ton of noise about this, but frankly, no one has actually defined what AGI means. I feel like the goal post is constantly shifting. Take for example Humanity's Last Exam. It is so broad and complex that while an individual in a specific field might be able to answer their specific area of questions, they certainly would not be able to achieve >50% on the total questi…

There's a specific singularity theory of AI that is very popular. Eliezer Yudkowsky helped popularize it among the Bay Area "rationalist" community, and it has this idea that AGI necessarily implies a self improving system that will quickly become a paperclip maximizer or other such dystopian or utopian world changing intelligence. By that singularity definition, we're probably nowhere near AGI, but if we define it a…

We'll probably be able to tell if and when Yudkowsky's AGI arrives.

Once AI can improve itself, frontier labs will no longer need human developers. So we might see a massive layoff of top talent and a dramatic increase in product quality at the same time. This usually doesn't happen in human businesses. It is also very much against the interest of anyone who is already at the top of the pay table at those labs.

Re: Some thoughts about Anthropic's new cryptanalysis results

#75

Post by notathrowaway51 »

KPGv2 wrote: post #60
Forgeties79 wrote: post #57

Earlier quoted context omitted.

> It's harder to believe something is conscious or threatening to achieve word domination once you understand that it's a machine that statistically figures out which word should come next. At the risk of sounding overly flippant, all world domination has been achieved by some person(s) figuring out which word should come next. Words quite literally = action when it comes to LLM’s with tools access

> At the risk of seeming X, statement that overwhelmingly demonstrates X-ness. It's exhausting to even consider where to begin addressing the assertion that good leadership is just predicting the next word to say. Especially considering the corpus available to most great leaders in history was extremely small. To think Hannibal's military campaigns were just because he'd read like ten books in his life and could accu…

"Competent strategy" is just as good a target for a word predictor to optimize as "effective rhetoric" is.

Re: Some thoughts about Anthropic's new cryptanalysis results

#76

Post by shard972 »

kijin wrote: post #74
annzabelle wrote: post #67

Earlier quoted context omitted.

There's a specific singularity theory of AI that is very popular. Eliezer Yudkowsky helped popularize it among the Bay Area "rationalist" community, and it has this idea that AGI necessarily implies a self improving system that will quickly become a paperclip maximizer or other such dystopian or utopian world changing intelligence. By that singularity definition, we're probably nowhere near AGI, but if we define it a…

We'll probably be able to tell if and when Yudkowsky's AGI arrives. Once AI can improve itself, frontier labs will no longer need human developers. So we might see a massive layoff of top talent and a dramatic increase in product quality at the same time. This usually doesn't happen in human businesses. It is also very much against the interest of anyone who is already at the top of the pay table at those labs.

What you describe would still be limited with so called AGI due to the existing organizational, legal and social structures around basic things like responsibility. We could have scifi level robots available today and these non-tech issues would still need to be resolved for a company to just fire all it's employees.

Re: Some thoughts about Anthropic's new cryptanalysis results

#77

Post by altmanaltman »

annzabelle wrote: post #67
sph87 wrote: post #65

Earlier quoted context omitted.

> AGI is already here I feel like there has been a ton of noise about this, but frankly, no one has actually defined what AGI means. I feel like the goal post is constantly shifting. Take for example Humanity's Last Exam. It is so broad and complex that while an individual in a specific field might be able to answer their specific area of questions, they certainly would not be able to achieve >50% on the total questi…

There's a specific singularity theory of AI that is very popular. Eliezer Yudkowsky helped popularize it among the Bay Area "rationalist" community, and it has this idea that AGI necessarily implies a self improving system that will quickly become a paperclip maximizer or other such dystopian or utopian world changing intelligence. By that singularity definition, we're probably nowhere near AGI, but if we define it a…

I think its fair to blame the AI labs for constantly hyping up the intelligence in a consiousness-related way where the model becomes "just like a human". Of course that is bs in terms that we have no clue on consiousness of the underlying models or even if its possible to do so.

But a purist defintion of AGI does not require consiousness as a part of it but can be seen as a benchmark metric. Sam Altman recently said the term doesn't really matter. And its true, even if AGI is achieved in the sense that a model can excel at all tasks at par or better than a human, then it is very useful but not as scary as a living, self-serving AI system like Skynet.

Yet AGI in the form discussed above will still grant massive power to AI labs if its injected into all domains. I recently wrote on this a bit on my blog: https://decodingvibes.com/blog/ai-can-ride-my-bike-with-no-h...

Re: Some thoughts about Anthropic's new cryptanalysis results

#78

Post by mirekrusin »

kijin wrote: post #74
annzabelle wrote: post #67

Earlier quoted context omitted.

There's a specific singularity theory of AI that is very popular. Eliezer Yudkowsky helped popularize it among the Bay Area "rationalist" community, and it has this idea that AGI necessarily implies a self improving system that will quickly become a paperclip maximizer or other such dystopian or utopian world changing intelligence. By that singularity definition, we're probably nowhere near AGI, but if we define it a…

We'll probably be able to tell if and when Yudkowsky's AGI arrives. Once AI can improve itself, frontier labs will no longer need human developers. So we might see a massive layoff of top talent and a dramatic increase in product quality at the same time. This usually doesn't happen in human businesses. It is also very much against the interest of anyone who is already at the top of the pay table at those labs.

Layoffs would be a poor indicator that recursive self-improvement had occurred as you need people with knowledge around domain/layer the work is done on.

Also, if the lab truly has a self-improving superintelligence, the cost of retaining staff at any level would be a rounding error relative to its operating costs and the value the system creates.

There would be little economic pressure to fire them immediately, especially while they remain useful for oversight, interpretation, risk management or simply as "interface" to the rest of the world etc.

If anything, they would probably hire more people to pursue more opportunities in parallel.

Re: Some thoughts about Anthropic's new cryptanalysis results

#79

Post by [deleted] »

annzabelle wrote: post #67
sph87 wrote: post #65

Earlier quoted context omitted.

> AGI is already here I feel like there has been a ton of noise about this, but frankly, no one has actually defined what AGI means. I feel like the goal post is constantly shifting. Take for example Humanity's Last Exam. It is so broad and complex that while an individual in a specific field might be able to answer their specific area of questions, they certainly would not be able to achieve >50% on the total questi…

There's a specific singularity theory of AI that is very popular. Eliezer Yudkowsky helped popularize it among the Bay Area "rationalist" community, and it has this idea that AGI necessarily implies a self improving system that will quickly become a paperclip maximizer or other such dystopian or utopian world changing intelligence. By that singularity definition, we're probably nowhere near AGI, but if we define it a…

[deleted]

Re: Some thoughts about Anthropic's new cryptanalysis results

#80

Post by probably_wrong »

dboreham wrote: post #12
simonw wrote: post #6

This is good: > If you’re under the impression that these models are “glorified autocomplete” or that progress is slowing down, I need to urge you: stop thinking that . The models are very intelligent and capable, they are getting better at a fast clip. I can cite measurable and impressive progress over just the past five months on specific types of problem I’ve asked them to look at. [...] > On the other hand : if y…

I'm also getting irritated with the “glorified autocomplete” comments. Since nobody can post such comments and also use the tools I'm using, I'm wondering if the phenomenon is due to people only having experience with the free version of whatever it is they're trying to use?

Let's say I'm one of such people. How would you convince me that your comment is any different from the ones we've been getting since roughly GPT-3 claiming that AGI is finally here and that, while the previous models were bad, this time they'll live up to the hype?
Post reply on HN