Live data from Hacker News

Statement on AI Risk

safe.ai

691–700 of 964 posts

Re: Statement on AI Risk

#691
post #646

Earlier quoted context omitted.

Are you really arguing ChatGPT is already super-intelligent? What is your basis for this conclusion? And many people argue against the idea that GPT is already super intelligent or even can become so at this stage of development and understanding. In fact as far as I can tell it is the consensus right now of experts and it's creators. https://www.calcalistech.com/ctechnews/article/nt9qoqmzz

If super means "surpassing normal human intelligence" then, YES. Take a look at the table in this article. If a human did that, was fluent in every language and coded in every language, we'd say they were superhuman, no? https://cdn.openai.com/papers/gpt-4.pdf

No. It’s not reasoning in any way. It’s an impressive parrot.

Re: Statement on AI Risk

#692
post #644

Earlier quoted context omitted.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

Maybe these people have good intentions and are just being naive They might not be getting paid, but that doesn’t mean they are not being influenced AI at this point is pretty much completely open, all the papers, math and science behind it are public Soon, people will have advanced AI running locally on their phones and watches So unless they scrub the Internet, start censoring this stuff, and pretty much ban comput…

You don't have to be a mustache-twirling villain to have the same effect.

Re: Statement on AI Risk

#694
post #510

Earlier quoted context omitted.

This is not how entropy works. The problem with talking about logical physical systems, is you have to understand the gradient against entropy. There are a trillion more ways to kill you then there are to keep you alive. There is only the tiniest sliver of states in which remain human and don't turn to chemical soup or physics. Any AI capable of it's power bill would be able to tell you that today, and that answer is…

Sure, but clever mumbo jumbo won't distract from the principle point. If AI can create a virus that kills all humans, another AI can create a virus that kills that virus. The virus has trillions more ways to be killed than to keep it alive, right?

That doesn't follow. It's like saying "if the AI can build a gun that can kill a human, it can build an anti-gun that can stop the gun".

There are lots of situations where offense and defense are asymmetrical.

So maybe the killer AI would need two months to build a time-delayed super-virus, and the defender AI would need two months to build a vaccine; if the virus takes less than two months to spread worldwide and activate, humanity is still dead.

Re: Statement on AI Risk

#696

This is a breathless, half-baked take on "AI Risk" that does not cast the esteemed signatories in a particularly glowing light. It is 2023. The use and abuse of people in the hands of information technology and automation has now a long history. "AI Risk" was not born yesterday. The first warning came as early as 1954 [1]. The Human Use of Human Beings is a book by Norbert Wiener, the founding thinker of cybernetics…

This topic clearly touches a nerve with the HN community, but I strongly agree with you.

To be honest, I've been someone disappointed with the way AI/DL research has proceeded in the last several years and none of this really surprises me.

From the beginning, this whole enterprise has been detached from basic computational and statistical theory. At some level this is fine — you don't need to understand everything you create — but when you denigrate that underlying theory you end up in a situation where you don't understand what you're doing. So you end up with a lot of attention paid to things like "explainability" and "interpretability" and less so to "information-theoretic foundations of DL models", even though the latter probably leads to the former.

If you have a community that considers itself above basic mathematical, statistical, and computational theory, is it really a surprise that you end up with rhetoric about it being beyond our understanding? In most endeavors I've been involved with, there would be a process of trying to understand the fundamentals before moving on to something else, and then using that to bootstrap into something more powerful.

I probably come across as overly cynical but a lot of this seems sort of like a self-fulfilling prophecy: a community constituting individuals who have convinced themselves that if it is beyond their understanding, it must be beyond anyone's understanding.

There are certainly risks to AI that should be discussed, but it seems these discussions and inquiries should be more open, probably involving other people outside the core community of Big Tech and associated academic researchers. Maybe it's not that AI is more capable than everyone, just that others are maybe more capable of solving certain problems — mathematicians, statisticians, and yes, philosophers and psychologists — than those who have been involved with it so far.

Re: Statement on AI Risk

#697
post #651

No signatories from Amazon or Meta. Also: they focus on extinction events (how are you gonna predict that?) but remain silent on all the ways that AI already sucks by connecting it to systems that can cause human suffering, e.g. sentencing[1]. My opinion: this accomplishes nothing, like most open letter petitions. It's virtue signaling writ large. [1] https://www.technologyreview.com/2019/01/21/137783/algorithm...

> AI already sucks

Not to mention what 'automation' or 'tech-driven capitalism' has already done to society over the past 100 years with effects on natural habitat and human communities. Stating 'AI risk' as a new risk sort of implies it's all been dandy so far, and suddenly there's this new risk.

Re: Statement on AI Risk

#698

Earlier quoted context omitted.

Could you be more specific about the conflicts you've uncovered?

It’s unequivocal that academics may have conflicts (in general), that’s why disclosures are required for publications. I’m not uncovering anything, several of the academic signatories list affiliations with OpenAI, Google, Anthropic, Stability, MILA and Vector resulting in a financial conflict. Note that conflict does not mean shill, but in academia it should be disclosed. To allay some concerns a standard disclosure…

I'm not really interested in doing a research project on the signatories to investigate your claim, and talking about things like this without specifics seems dubiously useful, so I don't really think there's anything more to discuss.

Re: Statement on AI Risk

#699

Earlier quoted context omitted.

Subtract OpenAI, Google, StabilityAI and Anthropic affiliated researchers (who have a lot to gain) and not many academic signatories are left. Notably missing representation from the Stanford NLP (edit: I missed that Diyi Yang is a signatory on first read) and NYU groups who’s perspective I’d also be interested in hearing. Not committing one way or another regarding the intent with this but it’s not as diverse an aca…

Even if it’s just Yoshua Bengio, Geoffrey Hinton, and Stuart Russell, we’d probably agree the risks are not negligible. There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. Many of whom do not work for those companies. We’re talking about nuclear war level risks here. Even a 1% chance should definitely be a…

Some of the academics who signed are either not doing AI research e.g climatologists, genomics, philosophy. Or they have Google connections that aren't disclosed. E.g. Peter Norvig is listed as Stanford University but ran Google Research for many years, McIlrath is associated with the Vector Institute which is funded by Google.

Re: Statement on AI Risk

#700

Earlier quoted context omitted.

> There are quite a few researchers from Stanford, UC Berkeley, MIT, Carnegie Mellon, Oxford, Cambirdge, Imperial College, Edinburg, Tsinghua, etc who signed as well. I know the Stanford researchers the most and the “biggest names” in LLMs from HAI and CRFM are absent. It would be useful to have their perspective as well. I’d throw MetaAI in the mix as well. Merely pointing out that healthy skepticism here is not ent…

LLM is already a misnomer. Latest versions are multimodal. Current versions can be used to build agents with limited autonomy. Future versions of LLMs are most likely capable of more independence. Even dumb viruses have caused catastrophic harm. Why? It’s capable of rapid self replication in a massive number of existing vessels. You add in some intelligence, vast store of knowledge, huge bandwidth, and some aid by ma…

This gets countered by running one (or more) of those same amazing autonomous agents locally for your own defense. Everyone's machine is about to get much more intelligent.
Post reply on HN