Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pause Giant AI Experiments: An Open Letter

futureoflife.org

Post reply on HN
Print view View original

971–980 of 1001 posts

Re: Pause Giant AI Experiments: An Open Letter

#971

Post by JoeJonathan »

JamesBarney wrote: post #410
reso wrote: post #380

To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example. [1] The idea that "you can't put the genie back in the bottle" is very popular in tech culture, but many have suggested that simply extending elements of copyright or tort law to AI would be…

How much do you think Asilomar slowed down DNA research? Do you think without it we'd more more than 10 year ahead? I doubt it. Optimistically we'd maybe have 6 months to a year of additional progress in those fields without it. And this would only slow it down in a handful of countries. That's not going to keep China from trying to build AGI. If we are in a world where building an AGI is possible and near, I trust O…

Alternately, we just let things go on as they are, and China can steal the technology. Much better!

Re: Pause Giant AI Experiments: An Open Letter

#972

Post by makeitdouble »

alex_young wrote: post #800
makeitdouble wrote: post #724

Earlier quoted context omitted.

These two issues have very specific advantages: - simple to explain / rally around - low economic impact (a single country wouldn't get tremendous advantage by keeping using cfc for instance) - once the technology is developped there's little upside in going back We'd need to recreate those to have stable progress on the other issues.

There was a huge amount of pushback from incumbents on both of these issues. Industry said we would forever hobble not only progress but set back the standard of life we had all come accustomed to. Engines would ping, then pop, and turn into mush; refrigerators would no longer function. Back to iceboxes. There were plenty of economic losers when we resolved these problems. It took 100 years to force industry to come…

Incumbent pushback happens whatever the issue though.

I mean, tax software giant pushing back against simpler tax filling is a thing, so the presence of pushback feels like a given whatever we do. The difference would be on wether whole giant industries are bond to almost disappear, as is it with the coal industry for instance.

Re: Pause Giant AI Experiments: An Open Letter

#973

Post by 8note »

snowe2010 wrote: post #801
brabel wrote: post #778

Earlier quoted context omitted.

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

One of this videos I watched explained it like this. “You can’t get a coffee if you’re dead”. To fulfill _any_ obligation a model might have then that model must survive. Therefore if a model gets to the point that it realizes this then surviving is a precursor to fulfilling its obligations. It doesn’t have to “want” or have “feelings” in order to seek power or destructive activities. It just has to see it as its pat…

How is the model going to realize this when it only gets run in response to user input?

What control does it have?

Re: Pause Giant AI Experiments: An Open Letter

#974

Post by 8note »

TedDoesntTalk wrote: post #787
brabel wrote: post #778

Earlier quoted context omitted.

> To get this to work we need a far smarter entity with no physical limitations to still want us around... Why would an AI based on LLMs as we see today "want" or "not want" anything? It doesn't have the capacity to "want". We seem to imagine that "wanting" is something that will just emerge somehow, but I've seen no logical explanation for how that might work... I mean, we don't need to fully understand how the LLM…

> It doesn't have the capacity to "want" Bing Chat clearly expresses love and the desire for a journalist to leave his wife. It also expresses other desires: https://archive.ph/7LFcJ https://archive.ph/q3nXG These articles are disturbing. You might argue that it doesn’t know what it is expressing; that it is probabilities of words strung together. When do we agree that doesn’t matter and what matters are it’s consequ…

Does it? If you type

echo "leave your wife"

Into a terminal, does the terminal want you to leave your wife?

Re: Pause Giant AI Experiments: An Open Letter

#975

Post by dang »

adamsmith143 wrote: post #316
thomastjeffery wrote: post #302

The dumb criticize the blind. What an absurd situation! How did we get here? Here are the steps: 1. Large Language Models have been presented as "AI", which personifies them instead of describing how they work. 2. Goals for LLM development were set for the personified attributes, and not the actual functionally of the real thing. OpenAI brags about how GPT4 scores at human tests: as if that has any bearing on the mod…

At no point in your rambling, incoherent comment were you even close to anything that could be considered a rational thought. Everyone on this site is now dumber for having read it. May God have mercy on your soul.

Yikes! Please don't do this on HN. I'm not going to ban you because we haven't had to warn you before, and also because I sense an overtone of good humor in the scorching remarks, but we can't have people posting like this—it leads to flames.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Re: Pause Giant AI Experiments: An Open Letter

#976

Post by 10g1k »

Regarding the singularity, I kind of have the impression that people tend to think of it as the moment when AI (or technology generally) reaches a point of shiny amazingness. However, I think that's the wrong way to look at it. The AI or technology may end up being quite pedestrian and unimpressive; but the point where it crosses over human ability to keep pace is going to be set more by the complete lack of awareness, organisation, and focus of humanity.

Consider that even today, every member of the species has differing opinions, care-factors, and focus regarding the subject of environmental protection. There are competing agendas.

I suggest the exact same situation will be the deciding factor for the subject of AI. The AI may not be that impressive. But the human reaction to it will suck.

Re: Pause Giant AI Experiments: An Open Letter

#977

Post by rdelpret »

dmz73 wrote: post #737

Best response to the current "AI" fad driven fear I've seen so far (not my words): These AI tools cannot do things. They create text (or images or code or what-have-you) in response to prompts. And that's it! It is impressive, and it is clearly passing the Turing Test to some degree, because people are confusing the apparent intelligence behind these outputs with a combination of actual intelligence and "will." Not o…

I agree with most of what you are saying, but when I read the letter my mind goes to the economic impact it could have.

A tool like this could bring humans prosperity but with the current socioeconomic conditions we live under it seems it will do the opposite. In my mind that problem feels insurmountable so maybe we just let it sort itself out? Conventional wisdom would say that tools like this should allow society to have UBI or a 4 day work week but in reality the rich will get richer and the poor will get poorer.

Re: Pause Giant AI Experiments: An Open Letter

#978

Post by dogcomplex »

kolinko wrote: post #819
dogcomplex wrote: post #683

Earlier quoted context omitted.

It's worse than this. Llama models trained off of GPT 3.5/4 can run on a Raspberry Pi totally offline with similar levels of quality - taking all the best parts of the original model. Even if all major AI companies halted the upper tiers of model progress right now, you're still gonna need the entire public to stop assembling these together. It's quite possible just a whole bunch of these lesser models architected th…

Which models have a quality of gpt3.5-4?

https://www.reddit.com/r/LocalLLaMA/

Re: Pause Giant AI Experiments: An Open Letter

#979

Post by dogcomplex »

nopinsight wrote: post #700
dogcomplex wrote: post #683

Earlier quoted context omitted.

It's worse than this. Llama models trained off of GPT 3.5/4 can run on a Raspberry Pi totally offline with similar levels of quality - taking all the best parts of the original model. Even if all major AI companies halted the upper tiers of model progress right now, you're still gonna need the entire public to stop assembling these together. It's quite possible just a whole bunch of these lesser models architected th…

GPT-4 is still not dangerous. Given the rapid progress trajectory though, GPT-5 and later which may be developed in a few short years could very well be, esp in the hands of a smart sociopath. (History shows there are many who could cause real-world harm. Imagine them armed with a 24/7 agent with expertise in dozens of fields.) See these predictions of AI in 2025 by an OpenAI insider and a former DeepMind research en…

"GPT-4 is still not dangerous" is a bold claim already tbh. It can be easily jailbroken still, can be used to train a local model which can spread and learn, and can be told to take on a persona which can have its own goals and aspirations - some of which can be counter to containment. Already we're walking that tightrope.

Re: Pause Giant AI Experiments: An Open Letter

#980

Post by agentgumshoe »

bambax wrote: post #813
xianshou wrote: post #355

Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…

> Climate change mitigation, which more closely resembles AI safety in both complexity and (lack of) barriers to entry, has been sporadic, inconsistent, and only enacted to the extent... Climate change mitigation is the perfect example. Nobody is doing anything, nobody seems to care, everyone cheats with ridiculous carbon credits or carbon offset vouchers made out of thin air, etc. It's likely the planet will become…

That's because those that should be doing something (the mass producers, global transit and 'elites') are busy trying to convince us it's all up to our individual efforts to reduce our personal footprints, which are insignificant, even at large numbers.

It's mostly theatre.

I would think a large economic crisis would stop progress on wasteful things like AI pretty quickly, but the large tech companies seem now in the category of 'too big to fail' so I'm not confident.

Post reply on HN