Statement on AI Risk • Hacker News Board

Re: Statement on AI Risk

#931

Post by Henk0 » Wed, May 31, 2023, 6:00 PM UTC

tech_ken wrote: post #633

dwaltrip wrote: post #198

Earlier quoted context omitted.

It’s not just random text, they are predicting writings and documents produced by humans. They are “language” models. Language is used to say things about the world. This means that predicting language extremely well is best done through acquiring an understanding of the world. Take a rigorous, well-written textbook. Predicting a textbook is like writing a textbook. To write a good textbook, you need to be an expert…

>Predicting a textbook is like writing a textbook HN AGI discourse is full of statements like this (eg. all the stuff about stochastic parrots), but to me this seems massively non-obvious. Mimicking and rephrasing pre-written text is very different from conceiving of and organizing information in new ways. Textbook authors are not simply transcribing their grad school notes down into a book and selling it. They are s…

A preemptive addendum to my previous reply:

Yes, LLM:s currently only deal with text information. But GPT-5 will supposedly be multimodal, so then it will also have visual and sound data to associate with many of the concepts it currently only knows as words. How many more modalities will we need to give it to be able to say that it understands something?

Also, GPT-4 indeed doesn't do any additional training in real-time. However, it is being trained on the interactions people have with it. Most likely, near future models will be able to train themselves continuously, so that's another step closer to how we function

Re: Statement on AI Risk

#932

Post by mola » Wed, May 31, 2023, 6:37 PM UTC

The "risks" from AI are far more mundane from this AGI taking over the world and extinction level event nonsense.

It's the usual, more power to the few, less power to the workers. Easy manipulation and control of the masses by "subtle" ways such as swamping the media with false divisive narratives (as being done today, but on steroids) and more violent ways such as automatic "police" without qualms or empathy.

The risks we face are from other humans.

This statement is at best naive misdirection. I won't be surprise if people like Altman are just working to increase their own power through regulatory capture.

This is a silicon valley power move. Some Randian heros with monarchic aspirations got some good folks to be their useful idiots.

Re: Statement on AI Risk

#933

Post by shrimpx » Wed, May 31, 2023, 6:55 PM UTC

NumberWangMan wrote: post #914

shrimpx wrote: post #702

Earlier quoted context omitted.

> slightly more advanced LLM I don't think there is a path, that we know if, from GPT4 to a LLM that could take it upon itself to execute complex plans, etc. Current LLM tech 'fizzles out' exponentially in the size of the prompt, and I don't think we have a way out of that. We could speculate though... Basically AI risk proponents make a bunch of assumptions about how powerful next-level AI could be, but in reality w…

When you have no clue, it makes sense to expand your confidence interval in both directions. So it could be a lot longer than we expect, but it could be shorter, too. You shouldn't just say "we have no clue, so it's probably further off and not worth worrying about". Especially since a lot of people made predictions about LLM capabilities and were surprised by how much better they worked than expected.

I agree in principle, it's just that the level of expressed worry doesn't seem to match reality. Currently we have no reasonable path to 'scary AGI'. It's some yet newfangled tech we haven't discovered.

As an example, consider the invention of motion picture. People were totally bewildered that you can have moving things and people inside a picture. Scaremongers could start claiming "Pretty soon the moving things may come to life and take over the world! Before you know it, they'll run our factories from inside the movies and we'll be their slaves!" That's more or less what this 'scary AGI' hype sounds like to me right now.

Btw "That Mitchell and Webb Look" is a great show ;-)

Re: Statement on AI Risk

#934

Post by tome » Wed, May 31, 2023, 7:33 PM UTC

pixl97 wrote: post #907

tome wrote: post #883

Earlier quoted context omitted.

OK, sure, in principle , somewhere in the universe, such a thing could arise. Why do you think there's a path to it arising on planet earth within human timescales?

Because humans keep pouring massive amounts of money into making this happen. When you invest dedicated effort in making something happen you greatly increase the probability that it happens. And in this case it is a reinforcing feedback loop. Better technology begots better technology. Intelligence begots more intelligence. Evolution has to take a random walk to get where it is, it doesn't necessarily have a goal pa…

Yeah, but they can't make it happen if it's impossible. If humans poured massive amounts of money into a perpetual motion machine I wouldn't expect it to happen. So what is it that makes you believe that artificial general intelligence is possible to get to at all?

Re: Statement on AI Risk

#935

Post by patch_cable » Wed, May 31, 2023, 7:46 PM UTC

patrec wrote: post #788

patch_cable wrote: post #730

Earlier quoted context omitted.

Reading the lesswrong link, the parts I get hung up on are that it appears in these doomsday scenarios humans lose all agency. Like, no one is wondering why this computer is placing a bunch of orders to DNA factories? Maybe I’m overly optimistic about the resilience of humans but these scenarios still don’t sound plausible to me in the real world.

> Like, no one is wondering why this computer is placing a bunch of orders to DNA factories? I'm not that confident that if we put you in a box, tron-style, where you basically continued to enjoy your existing level of intelligence, but think 10'000x faster, have Petabytes of information at your fingertips and can clone yourself and losslessly and rapidly exchange knowledge with your clones and had a few days to thin…

Maybe that’s the difference in our views, but yes I am confident.

Re: Statement on AI Risk

#936

Post by frankzinger » Wed, May 31, 2023, 7:49 PM UTC

computerphage wrote: post #886

frankzinger wrote: post #864

Earlier quoted context omitted.

That link is about the risks of AGI, which doesn't exist, and there's no reason to believe that it ever will exist. (If I'm wrong about AGI then I'm open to being convinced, but that's a different conversation as the topic here is non-general AI, is it not?)

I disagree that there's no reason to believe it will ever exist. For one thing, many smart people are trying to build the technology right now and they believe it to be possible. I see no compelling case that the intelligence scale simply tops-out with humans; that a more intelligent system is ruled out by the laws of physics. The topic here is human extinction caused by AI. I don't know of any serious argument for w…

Maybe these AI Apocalypse articles published for general consumption would be justified if there were any signs whatsoever that we were on a path towards AGI but there are none, are there? Even the best we have today are still just machines. They are clearly not really intelligent. At best they simulate intelligence, but poorly (because they still make ridiculous mistakes). Just because there are no physical limits to intelligence doesn't mean it's possible for beings with finite intelligence to create infinite intelligence. It all just seems extremely premature to me.

Re: Statement on AI Risk

#937

Post by revelio » Wed, May 31, 2023, 9:12 PM UTC

gjm11 wrote: post #923

revelio wrote: post #899

Earlier quoted context omitted.

I thought someone might say that. This thread is really a perfect demonstration of my point. Our society is so in thrall to self-proclaimed intellectuals that you can literally make a movie presenting falsifiable claims with 100% confidence, people can say at the time "this is absurd and will not happen", you can spend years attacking those critics, it can then not happen and still you will have an army of defenders…

Well, obviously what I say must be wrong if you guessed that someone might say it. Indeed, someone might say "95%" because they want to make the same sort of impression as if they said "100%" but to be able to hide behind it if they're wrong. Or, y'know, they might say "95%" because they've thought about the strength of the evidence and expect to be right about such things about 95% of the time. (I'm not sure how rel…

There's a ton of stuff here and this isn't a climate thread, but briefly:

1. Yes I believe independents are more reliable than full time researchers because the latter are deeply conflicted and independents aren't.

2. They don't work for the oil industry. I've checked. That's propaganda designed to stop people listening.

3. There was in fact a very real pause, not simply due to statistical chance. Climatologists didn't predict that and fixed the problem by altering the historical record to erase it. That's why you can't see a pause now - not because there wasn't one, but because any time temperature graphs don't go according to plan, they change the databases with the measurements so they do. Look into it. There's another pause going on right now! In fact temperatures seem to have been stable for about 20 years modulo an El Nino in ~2015, which is natural.

4. A big part of why they're unreliable is that these guys don't engage with the real world. A major embarrassment for them was when people started driving around and looking at the actual ground station weather stations and discovered what an unusable data trash fire the ground network was - this was something climatologists themselves hadn't bothered to ever look at! You'd expect these experts to know more about the quality of their data than random bloggers but it wasn't so.

5. Where do you think Al Gore got his original claims? He didn't invent them out of whole cloth. They came from climatologists, of course.

You can go back and forth on climate related claims all day and get nowhere because the field is so corrupt that half the data is truncated, manipulated, tipped upside down, cherry picked, or wholesale replaced with the output of models whilst being presented as observation. It should go without saying but if the people who control the data also make the predictions, then they will never be wrong regardless of what happens in reality!

Re: Statement on AI Risk

#938

Post by gjm11 » Wed, May 31, 2023, 10:17 PM UTC

revelio wrote: post #937

gjm11 wrote: post #923

Earlier quoted context omitted.

Well, obviously what I say must be wrong if you guessed that someone might say it. Indeed, someone might say "95%" because they want to make the same sort of impression as if they said "100%" but to be able to hide behind it if they're wrong. Or, y'know, they might say "95%" because they've thought about the strength of the evidence and expect to be right about such things about 95% of the time. (I'm not sure how rel…

There's a ton of stuff here and this isn't a climate thread, but briefly: 1. Yes I believe independents are more reliable than full time researchers because the latter are deeply conflicted and independents aren't. 2. They don't work for the oil industry. I've checked. That's propaganda designed to stop people listening. 3. There was in fact a very real pause, not simply due to statistical chance. Climatologists didn…

1. Noted. (I don't think this is a strong effect and think it is outweighed by the fact that the full-time researchers know and understand more.)

2. I said "funded by" rather than "employed by". For instance, consider James Taylor of the Heartland Institute, mentioned above. He doesn't "work for the oil industry" in the sense of being on the payroll of an oil company. (So far as I know, anyway.) But the Heartland Institute, before it decided to stop disclosing its sources of funding, took quite a bit of money from ExxonMobil and at least some from the Koch Foundation. (Also Philip Morris, of course; before the Heartland Institute got into casting doubt on the harms of fossil fuels, it was into casting doubt on the harms of tobacco smoking.) Ross McKitrick is a senior fellow of the Fraser Institute (funded by, among others, the Koch Foundation and ExxonMobil) and is on the board of the Global Warming Policy Foundation, which claims not to take funding from people with connections to the energy industry but takes plenty from Donors Trust (an entity that exists, so far as I can tell, solely to "launder" donations between right-wing organizations so that e.g. the Koch Foundation can send money to the GWPF without the GWPF literally explicitly getting it from the Kochs) and other entities with substantial ties to the fossil fuel industry.

None of which, again, is literally "on the payroll of fossil fuel companies". If you find that that's enough to stop you being bothered by the connections, that's up to you; I am not quite so easily reassured.

3. I would be interested in details of this alleged falsification of the historical record. The graph looks to me exactly like what you get if you combine a steady increasing trend with seasonal oscillation (El Nino) and random noise. After a peak in the seasonal oscillation it looks like the warming has slowed for some years. Then it looks like it's going much faster than trend for some years. If you can look at the graph I pointed you at and say with a straight face that the people saying in ~2010 that "global warming has stopped" were anything like correct, then I'm really not sure what to say to you.

Anyway, I'm going to leave it here. I don't get the impression that further discussion is very likely to be fruitful.

Re: Statement on AI Risk

#939

Post by patrec » Wed, May 31, 2023, 11:42 PM UTC

patch_cable wrote: post #721

patrec wrote: post #682

Earlier quoted context omitted.

OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-e…

I think between point 3 and 4 there is a leap to talking about “danger”. Perhaps the disagreement is about what one calls “danger”. I had perhaps mistakenly assumed we were talking about an extinction risk. I’ll grant you concerns about scaling up things like surveillance but there is a leap to being an existential risk that I’m still not following.

The default assumption ought to be that anything that's very fast, very smart, goal-directed, self-improving, and self-replicating spells trouble, no?

Re: Statement on AI Risk

#940

Post by patrec » Wed, May 31, 2023, 11:49 PM UTC

csomar wrote: post #858

patrec wrote: post #58

Earlier quoted context omitted.

> It’s hard to take these safety concerns seriously I don't get this mindset at all. How can it not be obvious to you that AI is an uniquely powerful and thus uniquely dangerous technology? It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of t…

> It's like saying nuclear missiles can't possibly be dangerous and nuclear arms reduction and non-proliferation treaties were a scam, because the US, China and the Soviet Union had positioned themselves to capture the majority of the strategic value nukes bring. I'm honestly not sure if this is sarcasm. The non-proliferation treaties are indeed a scam. The war is raging between the US and Russia and nuclear is a big…

Not only is the Non proliferation treaty possible, it's been evidently effective in slowing the proliferation of nuclear weapons. The only country that ratified or acceded to it and went on to develop nuclear weapons is North Korea, and the only country that ratified or acceded to it and looks on track to develop nuclear weapons is Iran. One country that was not a signatory and developed nuclear weapons voluntarily gave them up and acceded to it partly due to international pressure (South Africa). Israel, Pakistan and India have since developed nuclear weapons but they were never signatories, the only other non-signatory is South Sudan which probably won't acquire nuclear capabilities anytime soon.