Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Statement on AI Risk

safe.ai

Post reply on HN
Print view View original

851–860 of 964 posts

Re: Statement on AI Risk

#851

Post by acjohnson55 »

timmytokyo wrote: post #428
acjohnson55 wrote: post #304

Earlier quoted context omitted.

Right now, he's openly calling for regulation. That's a verifiable fact. It's very possible that when specific proposals are on the table, that we'll see Altman become uncooperative with respect to things that don't fit into his self-interest. But until that happens, you're just speculating.

He's not saying "please regulate me!" He's saying "Please regulate my competitors!" I see no reason to assume benign motives on his part. In fact there is every reason to believe the opposite.

It's simply not correct that he is asking only for his competitors to be regulated, and it's extremely well documented. https://www.bbc.com/news/world-us-canada-6561686

Re: Statement on AI Risk

#852

Post by calf »

h___ wrote: post #673
ChatGTP wrote: post #66

Earlier quoted context omitted.

If you were to take a look at the list of signatories on safe.ai, that's basically everyone who is everyone that works on building AI, what could Emily B Bender a professor of computer linguistics possibly add to the conversation and how would she be able to talk more about the "real AI safety" than any of those people? Edit: Sorry if it sounds arrogant, I don't mean Emily wouldn't have anything to add, but not sure…

She’s contributed to many academic papers on large language models and has a better technical understanding of how they work and their limitations than most signatories of this statement, or the previous widely hyped “AI pause” letter, which referenced one of her own papers. Read her statement about that letter ( https://www.dair-institute.org/blog/letter-statement-March20... ) or listen to some of the many podcasts…

Thanks for the link, I read it with interest but philosophically it is a flawed argument (IMHO). It's a nonsequitur or something, for the following two reasons.

First, I'm inclined to think that longtermism is an invalid and harmful ideology, but also acknowledge that AGI / existential risk is something that needs looking at seriously. The external factors, such as corporate interests and 1% wealthy interests/prejudices, are not a good reason to dismiss AGI concerns. I'd like to imagine there's a reasonable way to address short-term issues as well as long-term issues. It's not an either-or debate.

Second, even from just a reading comprehension level: one side says AGI is a problem, then the other side cannot just say, "No, AGI is a false problem and here are the real problems". The reasonable argument is to say, AGI is a false problem because . Bender et. al are just sidestepping the moot point, and rhetorically this is not an okay move. I think honest experts could simply say, ultimately we don't really know what will happen. But that would be boring to say because it would require acknowledging multiple issues being valid.

(There's a well known chart, the hierarchy of disagreements. The most sophisticated intellectual disagreements point out what's wrong with the argument. Less sophisticated disagreements do things like point out alternatives, without pointing out what the critical mistake is. The critical mistake in this case hinges on whether the premise of AGI is true or not. That's the crux of disagreement. Substituting that with short-term issues, which are valid in themselves, are the example of a lower-level of argumentation. And of course even lower levels of argumentation are bad-faith readings and so forth, I forget but the chart had several levels. It's funny that professional academics nevertheless don't practice this and so get into endless, intellectually unsatisfactory debates.)

So I think this is actually an example of different factional experts constantly talking past each other. It's funny that famous intellectuals/experts constantly do this, let their egos get the better of themselves and having a real intellectual conversation rather than make basic debate mistakes like nonsequiturs that any college student should be able to point at.

Re: Statement on AI Risk

#853

Post by NovaDudely »

cwkoss wrote: post #807
NovaDudely wrote: post #798

Earlier quoted context omitted.

I think it is because anything that can be used for good can also be used for bad. Advances in gene editing can provide a miracle medicine or a new means for biological warfare. AI is the same, we can use it to do some great things but it can also be leveraged by bad actors and very easily. The broad scale of what it can be implemented on means there is a lot of attach surface for change in either direction. The AI s…

I think the only defense against superintelligent AI doom is superintelligent AI thinking about how to prevent AI doom. Fewer projects with concentrated control is thus more dangerous than a diversity of projects: the majority of AI wont want to destroy the planet or all humans, and will thus fight the ones that do.

I wouldn't say that very many, if any projects would set out to destroy the world. The vast majority of people are out to set the world right even if that view is wildly different depending on the perspective. By that I mean a terrorist is seen as a freedom fighter depending on the angle.

What I fear is that something innocuous will have wildly intended outcomes. A good example, it is thought that a part of the 2008 credit crash was caused by algorithms handling all the bundling of securities. In bundling bad debt with good, it could hide a lot of the risk in the system.

It comes down to the whole issue of trying to define the environment in which we deploy these things, defining the goals and hoping there are no gaps that the system can exploit in optimizing for outcome.

Re: Statement on AI Risk

#854

Post by rad88 »

adamsmith143 wrote: post #719
stevenhuang wrote: post #711

Earlier quoted context omitted.

More appropriate is an expected value approach. That is, despite it being a very low probability event, it may still be worth remediation due to the outsized negative value if the event does happen. Many engineering disciplines incorporate safety factors to mitigate rare but catastrophic events for example. If something is maximally bad, then it necessitates some deliberation on ways to avoid it, irrespective how unl…

Exactly. Taken to the limit if you extrapolate how many future human lives could be extinguished by a runaway AI you get extremely unsettling answers. Like the expected value of a .01% change of extinction from AI might be Trillions of quality Human lives. (This could in fact be on the very very conservative side, e.g. Nick Bostrom has speculated that there could be 10^35 human lives to be lived in the far future whi…

Actual people are vastly outnumbered by entropy fluctuations in the long run, according to any number of valid cosmological models. Their total population is greater than 10^35 by far, and does not depend on whether superintelligence is possible or likely to destroy the actual human race. That question makes no difference to almost every human who will ever have a thought or feeling.

You could say, well, that's just a theory -- and it is a dubious one, indeed. But it's far more established by statistics and physical law than the numbers Bolstrom throws out.

Re: Statement on AI Risk

#855

Post by calf »

derbOac wrote: post #696
nologic01 wrote: post #677

This is a breathless, half-baked take on "AI Risk" that does not cast the esteemed signatories in a particularly glowing light. It is 2023. The use and abuse of people in the hands of information technology and automation has now a long history. "AI Risk" was not born yesterday. The first warning came as early as 1954 [1]. The Human Use of Human Beings is a book by Norbert Wiener, the founding thinker of cybernetics…

This topic clearly touches a nerve with the HN community, but I strongly agree with you. To be honest, I've been someone disappointed with the way AI/DL research has proceeded in the last several years and none of this really surprises me. From the beginning, this whole enterprise has been detached from basic computational and statistical theory. At some level this is fine — you don't need to understand everything yo…

Do you have any stories of how AI/DL has ignored foundational scientific problems?

I do know that my old EECS professors who have pivoted toward AI and coming from adjacent/tangential research areas, are specifically interested in theoretical and scientific clarification properties of neural networks. One of them basically has been trying to start Theoretical Machine Learning as a new discipline and approach that is sorely needed.

Re: Statement on AI Risk

#856

Post by csomar »

chefandy wrote: post #487
AlexandrB wrote: post #303

This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.

I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…

Be aware that the things that AI challenges most is knowledge work. No food delivery boy job is being challenged here (who cares about these people anyway?) but if you are a software developer the clock on that is ticking.

Re: Statement on AI Risk

#858

Post by csomar »

patrec wrote: post #58
valine wrote: post #17

I have yet to see a solution for “AI safety” that doesn’t involve ceding control of our most powerful models to a small handful of corporations. It’s hard to take these safety concerns seriously when the organizations blowing the whistle are simultaneously positioning themselves to capture the majority of the value.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

> It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of the strategic value nukes bring.

I'm honestly not sure if this is sarcasm. The non-proliferation treaties are indeed a scam. The war is raging between the US and Russia and nuclear is a big part of it (though just words/threats for now). It's nonsensical to think that these treaties are possible.

Re: Statement on AI Risk

#859

Post by dragonwriter »

calf wrote: post #852
h___ wrote: post #673

Earlier quoted context omitted.

She’s contributed to many academic papers on large language models and has a better technical understanding of how they work and their limitations than most signatories of this statement, or the previous widely hyped “AI pause” letter, which referenced one of her own papers. Read her statement about that letter ( https://www.dair-institute.org/blog/letter-statement-March20... ) or listen to some of the many podcasts…

Thanks for the link, I read it with interest but philosophically it is a flawed argument (IMHO). It's a nonsequitur or something, for the following two reasons. First, I'm inclined to think that longtermism is an invalid and harmful ideology, but also acknowledge that AGI / existential risk is something that needs looking at seriously. The external factors, such as corporate interests and 1% wealthy interests/prejudi…

> First, I’m inclined to think that longtermism is an invalid and harmful ideology

It is, but (aside from as a sort of sociological explanation of the belief in AGI risk) that’s mostly beside the point when discussing the problems AGI x-risk. The problem of AGI x-risk is that it is an entirely abstract concern which does not concrete flow from any basis in material reality, cannot be assessed with the tools of assessing material reality, and exists as a kind of religious doctrine surrounded by rhetorical flourishes.

> The external factors, such as corporate interests and 1% wealthy interests/prejudices, are not a good reason to dismiss AGI concerns.

They are way of understanding why people who seem (largely because they are) intelligent and competent are trying to sell such hollow arguments as AGI x-risk. They aren’t, you are correct, a logical rebuttal to AGI risk, nor are they intended as that; the only rebuttal is the complete absence of support for the proposition. They are, however, a tool that operates outside the realm of formalized debate that addresses the natural and useful cognitive bias that itself is outside of the realm of formalized debate that says “smart, competent people don’t tend to embrace hollow positions”.

> Second, even from just a reading comprehension level: one side says AGI is a problem, then the other side cannot just say, “No, AGI is a false problem and here a the real problems”.

1. If they couldn’t, it wouldn’t be a “reading comprehension issue”, and

2. They can, for the simple reason that there is no material support for the “AGI is a real problem” argument.

> Bender et. al are just sidestepping the moot point,

A point being moot in the sense that AGI x-risk is is a reason to sidestep it. (The danger of using auto-antonyms.)

> I think honest experts could simply say, ultimately we don’t really know what will happen.

To the extent that is accurate, that is exactly what the Bender/Gebru/Mitchell group does. The problem is thinking “we don’t have any information to justify any belief on that” is one sided and means that utility of AGI is somewhere between 0 and the negative infinity that the x-risk crowd calculates from (some unspecified non-zero finite probability) times (infinite cost), whereas the reality is that we have as much reason to believe that AGI is the only solution to an otherwise certain existential calamity as to suppose it will lead to one. The utility is somewhere between positive infinity and negative infinity.

Re: Statement on AI Risk

#860

Post by joshuamorton »

comp_throw7 wrote: post #829
joshuamorton wrote: post #741

Earlier quoted context omitted.

You're putting a weirdly large amount of trust into, functionally, some dude who posted on lesswrong. Sure he has a PhD and is smart, but so is basically everyone else in the field , not just in alignment, and the median person in the field thinks the risk of "doom" is 2-5% (and that's conditioned on the supposed existence of a high level machine intelligence that the median expert believes might exist in 40 years).…

The median person in field thinks 5-10%, not 2-5%, and median timelines are shorter than 40 years. But this is all a distraction, since unaligned ASI is the default case absent significant efforts (that we aren't currently making), and trying to evaluate risk by averaging out the views of people who haven't actually explored the arguments very carefully (vs. just evaluating the arguments yourself) is a doomed effort.

> The median person in field thinks 5-10%, not 2-5%

The median person in the study here, under a particular definition was 5-10%, other comparable studies have found 2%, and similar questions using arguably better definitions in the same study found lower percentages.

> median timelines are shorter than 40 years.

The median person suggested a 50% chance in 39 years.

> since unaligned ASI is the default case

I challenge this assertion. Many relatively smart scholars who are more involved in the alignment space than, presumably either you or I, have put forth cogent arguments that alignment-by-default is perfectly reasonable. Dismissing those out of hand seems naive.

Post reply on HN