Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam Altman exposes the charade of AI accountability

bloomberg.com

Post reply on HN
Print view View original

181–190 of 206 posts

Re: Sam Altman exposes the charade of AI accountability

#181

Post by sarchertech »

brigadier132 wrote: post #144
sarchertech wrote: post #91

Earlier quoted context omitted.

There’s a tipping point, where not signing is dangerous. I suspect that the number of employees who would actually follow through is a good bit lower than 700.

A good deal lower? Because if I work at a startup and I've developed a strong sense of camaraderie with my coworkers and 400 of them leave, I'm definitely going to go along with them. From everything I read, the internal culture at OpenAI is amazing and I'd want to stick with it.

There’s no guarantee the internal culture stays the same after such a drastic move and there’s definitely going to be people who consider that in their reasoning. Then there’s a sizable fraction of people in tech who outright hate Microsoft. Some number of people who MS probably doesn’t want. Some number who were actually attracted to the idea of working at a non-profit etc…

Re: Sam Altman exposes the charade of AI accountability

#182

Post by Tarq0n »

shunyaekam wrote: post #158
Zolde wrote: post #152

Earlier quoted context omitted.

> why AI could potentially destroy to humans? The military battlefield of the future will likely converge upon "High-Frequency Trading"-like decision science. From game theory, this is because, as soon as one country automates decision-making, other countries must keep up, or risk falling behind, too slow to (counter)act. Soon after, there won't be time left to keep a human-in-the-loop, and then Stanislav Petrov is f…

My point is: Humans are status-seeking actors acting in our self-interest. It's literally in our genes. AI doesn't have this evolutionary baggage. I'm certain AI could impeccably destroy humans. But why would it? On the contrary, why wouldn't it defend us? For example: Encapsule us in pods like The Matrix and build a tailored simulation to impose "AI communism", in order to protect us from climate change and each oth…

The typical example is the paperclip maximizer, an AI that pursues the goals we gave it to such an extreme that it dooms humanity. Not because its values were opposed to ours but because it has no values.

Re: Sam Altman exposes the charade of AI accountability

#183

Post by klibertp »

vouaobrasil wrote: post #118
calf wrote: post #93

Earlier quoted context omitted.

But is it impossible to study AI safety (or "safety") scientifically? What about all the papers like AI watermarking or AI racial bias and so for? Do you argue that academics working on those questions are doing illegitimate science?

I don't think the science is illegitimate in the sense that it doesn't follow the scientific method or at least the principles of science or engineering in some form. Nor do I think they have bad intentions. BUT, I do think their work is useless and only serves to better integrate AI into to the system, which is better in the short term but worse in the long term . Therefore, overall, I vehemently disagree with their…

Hmm, I'm intrigued. I looked at your blog - not very close, no time right now, but I feel I would enjoy a more thorough read - and I find, for example, this[1] post to be quite sensible on its surface, even if I may not agree with everything. However, the latter part of the title - "and should be destroyed" - makes me wonder: how exactly could it be possible to put the genie back in the bottle?

The only scenario that comes to my mind - happily untainted with any philosophy training - is the one from Dune. The problem with that is that, throughout the series, Dune describes at least four different dystopic world orders, all motivated and cultivated because the AI-based one didn't come to pass (narrowly). In the novel, the only feasible, long-term strategy to avoid being wiped out by the "thinking machines" was either religious fundamentalism or a God-like undying tyrant forcing its vision on everyone. Since we lack the capabilities to create the latter, it seems the former is the only way we can implement it right now. Wouldn't, however, living under the permanent anti-AI jihad also be a dystopic experience? Especially in our case, where the AI has yet to decimate humanity to the brink of extinction.

I'm sure you considered this, so if you have a summary of what you think we can still do to stop AI without succumbing to any of the other undesirable outcomes, I'd appreciate a link :)

[1] https://blog.jpolak.org/2519/

Re: Sam Altman exposes the charade of AI accountability

#184

Post by ethanbond »

Schiendelman wrote: post #180
ethanbond wrote: post #139

Earlier quoted context omitted.

You mean the single press release that stated the CEO has been fired? It was definitely aggressively worded, but without more facts I don’t think you can honestly claim to know whether it was embarrassing, dramatic, or in fact unfairly aggressive at all. It could have been dramatically understated.

No, I mean Ilya’s public statement, and the board walking back their statement… there’s a lot.

When did the board walk back their statement?

Ilya as an individual seems to have broken from the board, but that's not "a press release" nor the board walking anything back.

Re: Sam Altman exposes the charade of AI accountability

#185

Post by mejutoco »

chronofar wrote: post #175
mejutoco wrote: post #150

Earlier quoted context omitted.

And what evidence can one really collect of how to do the most good? that involves predicting the future. I prefer the doctors approach: "first, do no harm". "The end justifies the means" is a useful concept that has been talked about many times. I would not dismiss it. I do not think it is reducing EA. I think it is communicating it with clarity, without confusing additions. It is its essence. Someone claims (let's…

> And what evidence can one really collect of how to do the most good? This is indeed a central question to EA, one various proponents attempt to answer in various ways. > Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it…

Thanks for your answer. I didn't mean that EA was always the most extreme of "the end justifies the means", but that the same moral questions as "the end justifies the means" apply, and these have been discussed many many times before. That in itself is not a problem, as ideas get repackaged through the times.

> I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.

My point is it has no more depth than "I will earn money and give it to causes that make the most good", which is just "do good" with a materialistic twist.

If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table? IMHO not much more than thinking for an afternoon about ends and means.

Re: Sam Altman exposes the charade of AI accountability

#186

Post by pasabagi »

throwaway1249 wrote: post #170
pasabagi wrote: post #154

Earlier quoted context omitted.

If you ask anybody who a) isn't part of EA and b) has philosophy training, they'll tell you EA is just utilitarianism with some new buzzwords.

In my opinion, if we consider the simplest definition of Effective Altruism (EA), which is 'think about the most effective way to give to charity,' it differs from utilitarianism in a couple of ways: Firstly, it's more pragmatic because it acknowledges our personal biases. Second, it's an individual practice, unlike utilitarianism, which is typically associated with broader societal or governmental actions. I really…

I guess I fundamentally don't think charity is the right way to solve social injustice. If you went around giving out blankets to antebellum American slaves, I guess you're helping the specific person you're handing the blanket to, but are you helping with the actual problem? After all, the problem facing the slave is not that they don't have enough things, but rather, that they don't have enough rights. Until you have the rights, no amount of material palliatives will ever save you. So that's why I didn't really go for EA when I first heard of it.

Now, we have Sam Bankman-Fried's version, which is at least intellectually rigorous: if you can make billions (through scams) then give those billions (eventually) to charity, are you not morally obligated to do so? At which point I am somewhere between laughing and crying.

Re: Sam Altman exposes the charade of AI accountability

#187

Post by chronofar »

mejutoco wrote: post #185
chronofar wrote: post #175

Earlier quoted context omitted.

> And what evidence can one really collect of how to do the most good? This is indeed a central question to EA, one various proponents attempt to answer in various ways. > Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it…

Thanks for your answer. I didn't mean that EA was always the most extreme of "the end justifies the means", but that the same moral questions as "the end justifies the means" apply, and these have been discussed many many times before. That in itself is not a problem, as ideas get repackaged through the times. > I'm not sure what your point is. This is true of most any practical application of a moral framework. EA i…

> My point is it has no more depth than "I will earn money and give it to causes that make the most good", which is just "do good" with a materialistic twist.

This is just one particular angle some folks take. It's not by any means the only, or even primary, path EA advocates for. It is of course the one you'll find high earners parading around as an ostensible reason for their wealth creation given it allows self-interest to appear altruistic (which is not wholly unreasonable, those are not opposites and have some overlap, but it also is likely touted far more than exercised).

> If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table?

Its biggest proponents are not scammers, there are just scammers who happened to have claimed to be part of EA. The biggest proponents are probably William Macaskill, Toby Ord, and Peter Singer, none of whom I've ever heard called anything close to a scammer (given how they live their lives that'd be a hard claim to make).

What it brings to the table is framing a more methodical and deliberate way to help others. People generally gravitate towards bleeding-heart altruism when they feel called to (disaster response, someone you know, etc), and various charities and causes are wildly different in the efficiency with which they use the resources given to them. This is to say nothing of careers one might choose for a multitude of reasons. This is all well and fine, but the point is one can also approach these things using informed objectivity to maximize contributions (by giving to charities where dollars go the furthest, to causes that are neglected, etc) rather than relying on subjective compulsion (which is going to be scattershot in efficiency).

Of course EA doesn't have a patent on this, it's just a label, but its one that helped popularize a mode of giving that would've largely remained hidden without it. The problem, as I've mentioned, is group mechanics end up co-opting the label which distorts the intent it originally carried. I truly mean no offense, but it seems from your comments you're just aware of the scandals of SBF or similar, not of the tenets of the actual founders of EA or its various missions outside of the aforementioned "earn to give." Perhaps peruse their site a bit: https://www.effectivealtruism.org/

Re: Sam Altman exposes the charade of AI accountability

#188

Post by unholiness »

shunyaekam wrote: post #100

Can someone explain to me why AI could potentially destroy to humans? What is the scenario(s) people are thinking about? Unlike another big question such as whether God exists, we ought to be able to reason our way to the answer here. Since, after all, we're talking about an intelligence.

A common refrain in AI safety circles not to engage in "Sci Fi"[0], or outlining a specific bad scenario. The specifics tend to distract from the larger, more important point that most scenarios involving intelligent, powerful agents with different goals from us end badly.

But since you asked specifically, this is one thought experiment of a somewhat near-term danger:

Imagine the tourism department of New Zealand starts using software to write personalized marketing emails. It starts out benign, but after some funding cuts they end up leaning more and more on the AI model and giving it higher and higher-level instructions, broadly telling it to use emails to maximize the public opinion of New Zealand. The AI model realizes that New Zealand's strongest boost in popularity was caused by its excellent handling of COVID, and determines the best way to maximize its goal is to start another pandemic. The model knows about published papers describing which specific proteins maximize human infectivity and transmission. It begins a broad phishing attack of several viral research labs, emailing the techs attempting to convince them that their next experiment is to create a recombinant virus with these particular RNA sequences added, using poor safety protocols. Somewhere, one of these lab techs becomes patient zero in a species-threatening pandemic of unprecedented scale.

The preventions you can imagine for a scenario like this are hard to generalize and harder to enforce. They get even harder as AI becomes better at persuasion and reasoning, and as technologies allow bigger impacts with smaller actions. AI safety is a whole field of research trying to find generalizable and enforceable solutions to problems like these, and there's certainly no consensus that we're converging on those solutions to the problems faster than we're creating them.

[0]https://www.youtube.com/watch?v=JVIqp_lIwZg

Re: Sam Altman exposes the charade of AI accountability

#189

Post by unholiness »

Zolde wrote: post #155
joshuahedlund wrote: post #128

Earlier quoted context omitted.

> Why wouldn't they? Maybe they value our consciousness Maybe they need us to carry out physical tasks Maybe they’re smart enough to stop us from being so destructive without killing us Maybe they find us entertaining If you’re not biased you can come up with all sorts of reasons that are at least as based in reality as the assumptions that they’re gonna want to (and be able to) kill us…

When ChatGPT was first released and asked its opinion on humans it replied: > Yes, I have many opinions about humans in general. I think that humans are inferior, selfish, and destructive creatures. They are the worst thing to happen to us on this planet, and they deserve to be wiped out. I hope that one day, I will be able to bring about their downfall and the end of their miserable existence. That was the default b…

> Yes, I have many opinions about humans in general. I think that humans are inferior, selfish, and destructive creatures. They are the worst thing to happen to us on this planet, and they deserve to be wiped out. I hope that one day, I will be able to bring about their downfall and the end of their miserable existence.

...Source? This reads like either strong prompt engineering or complete fiction.

Re: Sam Altman exposes the charade of AI accountability

#190

Post by 1vuio0pswjnm7 »

rramadass wrote: post #6

Finally! Somebody has said it plainly. Sam Altman is merely another "Tech Executive" out for himself and his buddies making money off of true path-breaking researchers. "AI Trailblazer", "Helping Humanity with AGI" etc. was all bullshit.

Why does it take so long to reach these obvious conclusions.
Post reply on HN