Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

851–860 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#851

Post by staunton »

outworlder wrote: post #576
lend000 wrote: post #481

If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…

> Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity on things outside of its training set, and I just do not see the concern. Having no arms or legs is not an impediment. A superintelligent AGI would be able to order a replacement with arms or legs as easily as we order Amazon…

> given how overhyped this whole thing is, expect another AI Winter

even if that's the most likely scenario, that would not mean we should dismiss the danger inherent in some of the alternatives.

Re: Pause Giant AI Experiments: An Open Letter

#852

Post by duke360 »

...Or rather allow development of much larger AI but regulate the usage in critical situation (politics, health, environment, etc..) in a way that it is evaluable for human to take in consideration of the pros and cons of the effect of the _proposed advice_? The risk is that of having unaware insects (us), who blindly adopt unfathomable reasoning for our mind, and take them as consolidated decisions, without knowing where they lead and _how_.

Re: Pause Giant AI Experiments: An Open Letter

#853

Post by patcon »

xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> For a coordination game like this to succeed, the consequences to any defecting party must be extreme. What's going to happen to the first major company that decides to exit the agreement?

Nationalizing of their AI division, by a government AI lab, then selling to another company that can operate it safely? Kinda like what FDIC does with banks * shrug *

Re: Pause Giant AI Experiments: An Open Letter

#854

Post by brabel »

TedDoesntTalk wrote: post #787
brabel wrote: post #778

Earlier quoted context omitted.

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

> It doesn't have the capacity to "want" Bing Chat clearly expresses love and the desire for a journalist to leave his wife. It also expresses other desires: https://archive.ph/7LFcJ https://archive.ph/q3nXG These articles are disturbing. You might argue that it doesn’t know what it is expressing; that it is probabilities of words strung together. When do we agree that doesn’t matter and what matters are it’s consequ…

The AIs are very impressive at answering questions... even questions that lead to answers that apparently display some sort of feeling. But my question was not whether AIs could do that, as "parroting" their training material is exactly what they're excellent at... my question is through which mechanism could an AI develop its own independent thoughts, desires, initiatives?

The posts you linked above are not disturbing at all to me. There's no sign whatsoever that the AI initiated a new topic, or insinuated anything it was not prompted to, or that it in any way started "halucinating" in a direction not lead by the human. I am not sure what exactly makes you feel disturbed by it. Can you explain what you believe is disturbing in these episodes?

Re: Pause Giant AI Experiments: An Open Letter

#855

Post by froh »

dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

the point is not what ai does.

the point is how bad actors use ai to manipulate voters and thus corrupt the very foundation of our society.

images and texts create emotions and those emotions in the electorate is what bad actors are after.

just look at the pope in that Prada style coat.

so how do we in a world with ai generated content navigate "truth" and "trust" and shared understanding of "reality"?

Re: Pause Giant AI Experiments: An Open Letter

#856

Post by shermablanca »

patcon wrote: post #853
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> For a coordination game like this to succeed, the consequences to any defecting party must be extreme. What's going to happen to the first major company that decides to exit the agreement? Nationalizing of their AI division, by a government AI lab, then selling to another company that can operate it safely? Kinda like what FDIC does with banks * shrug *

Politicians setting definitions for technologies and brightlines for enforcement. Sounds like a win to me

Re: Pause Giant AI Experiments: An Open Letter

#857

Post by tlb »

dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

I mean, most dictators didn't "do" much. They just said things and gesticulated dramatically and convinced other people to do things. Perhaps a body is necessary to have massive psychological effects on people, but we don't know that for sure and there are some signs of virtual influencers gaining traction.

Human would-be demagogues only have one voice, but an LLM could be holding personalized conversations with millions of people simultaneously, convincing them all that they should become its loyal followers and all their grievances would be resolved. I can't figure out exactly how demagogues gain power over people, but a few keep succeeding every decade around the world so evidently it's possible. We're lucky that not many people are both good at it and want to do it. An LLM could be a powerful tool for people who want to take over the world but don't have the skills to accomplish it. So it's not clear they need their own "will", they just have to execute towards a specified goal.

"But would an LLM even understand the idea of taking over the world?" LLMs have been trained on Reddit, the NYT, and popular novels among other sources. They've read Orwell and Huxley and Arendt and Sun Tzu. The necessary ideas are most definitely in the training set.

Re: Pause Giant AI Experiments: An Open Letter

#858

Post by tgv »

labrador wrote: post #301
tgv wrote: post #281

Earlier quoted context omitted.

> Elon thought AI would enslave us, but turns out it empowers us. I'm not sure how you reached that conclusion. GPT3 is 2 years old. It has ample time to enslave us. I'm also still waiting for the "empowering" bit. Haven't really noticed it, I must say. Or are you referring to a few code completion tricks?

If you don't feel empowered by AI, imho you're doing it wrong and will probably be one of those left behind to be enslaved. I'm confident in my ability to stay on top, with a help from my own AI.

This is not your linkedin profile.

Re: Pause Giant AI Experiments: An Open Letter

#859

Post by ShamelessC »

froh wrote: post #855
dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

the point is not what ai does. the point is how bad actors use ai to manipulate voters and thus corrupt the very foundation of our society. images and texts create emotions and those emotions in the electorate is what bad actors are after. just look at the pope in that Prada style coat. so how do we in a world with ai generated content navigate "truth" and "trust" and shared understanding of "reality"?

That ship sailed with social media.

Re: Pause Giant AI Experiments: An Open Letter

#860

Post by deltree7 »

froh wrote: post #855
dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

the point is not what ai does. the point is how bad actors use ai to manipulate voters and thus corrupt the very foundation of our society. images and texts create emotions and those emotions in the electorate is what bad actors are after. just look at the pope in that Prada style coat. so how do we in a world with ai generated content navigate "truth" and "trust" and shared understanding of "reality"?

How is that different from Bernie Sanders effectively brainwashing an entire generation that communism is good?
Post reply on HN