Earlier quoted context omitted.
I didn't see your other post I think, but here's my response to the list of AI risks on the website we're discussing: https://news.ycombinator.com/item?id=36123082#36129011 Arguments like yours are very subjective. What is "rapid". What is "surprising". I don't find them particularly surprising myself - cool and awesome - but I was being amazed by language modelling ten years ago! The quality kept improving every yea…
> Arguments like yours are very subjective. What is "rapid". What is "surprising". https://twitter.com/heyBarsee/status/1654825921746989057 2/3 deep learning Turing Price winners (Hinton and Benigo) are sufficiently shell-shocked by the rate of progress to be thrown into existential doubts (Hinton is very explicit about the fact that progress is much faster than he thought just a few years ago, Benigo speaks of how a…
Statement on AI Risk
951–960 of 964 posts
Re: Statement on AI Risk
#952https://www.soroushjp.com/2023/06/01/yes-avoiding-extinction...
Re: Statement on AI Risk
#953This reeks of marketing and a push for early regulatory capture. We already know how Sam Altman thinks AI risk should be mitigated - namely by giving OpenAI more market power. If the risk were real, these folks would be asking the US government to nationalize their companies or bring them under the same kind of control as nukes and related technologies. Instead we get some nonsense about licensing.
https://www.soroushjp.com/2023/06/01/yes-avoiding-extinction...
Re: Statement on AI Risk
#954Earlier quoted context omitted.
The last gas I passed had 10^23 entropy fluctuations in it. So you're saying those are more important than the 100 Billion human lives that came before them?
I'm saying that it's unscientific nonsense. Bolstrom has predicted that the number of people potentially in the future is between approximately 0 and 100,000,000,000,000,000,000,000,000,000,000,000. This isn't an estimate that can or should factor into any decision at all. 100B people are going to live in the 21st century. Worry about them.
Re: Statement on AI Risk
#955Earlier quoted context omitted.
I'm eternally skeptical of the tech business, but I think you're jumping to conclusions, here. I'm on a first-name basis with several people near the top of this list. They are some of the smartest, savviest, most thoughtful, and most principled tech policy experts I've met. These folks default to skepticism of the tech business, champion open data, are deeply familiar with the risks of regulatory capture, and don't…
It’s worth noting also that many academics who signed the statement may face adverse issues like reputational risk as well as funding cut to their research programs if AI safety becomes an official policy. For a large number of them, these risks are worth far more than any possible gain from signing it. When a large number of smart, reputable people, including many with expert knowledge and little or negative incenti…
Re: Statement on AI Risk
#956Earlier quoted context omitted.
I didn't see your other post I think, but here's my response to the list of AI risks on the website we're discussing: https://news.ycombinator.com/item?id=36123082#36129011 Arguments like yours are very subjective. What is "rapid". What is "surprising". I don't find them particularly surprising myself - cool and awesome - but I was being amazed by language modelling ten years ago! The quality kept improving every yea…
> Arguments like yours are very subjective. What is "rapid". What is "surprising". https://twitter.com/heyBarsee/status/1654825921746989057 2/3 deep learning Turing Price winners (Hinton and Benigo) are sufficiently shell-shocked by the rate of progress to be thrown into existential doubts (Hinton is very explicit about the fact that progress is much faster than he thought just a few years ago, Benigo speaks of how a…
To clarify my own thinking here, it's totally reasonable to me that people are surprised if:
1. They weren't previously aware of AI research (surely 99% of the population?)
2. They were but had stopped paying attention because it was just a long series of announcements about cool tech demos nobody outside big corps could play with.
3. They were paying attention but thought scaling wouldn't continue to work.
My problem is that people like Sam Altman clearly aren't in any of those categories and Hinton shouldn't have been in any, although maybe he fell into (3). I personally was in (2). I wasn't hugely surprised that ChatGPT could exist because I'd seen GPT-2, GPT-1, I'd seen surprising AI demos at Google years earlier and so on. The direction things were going in was kinda clear. I was a bit surprised by its quality, but that's because I wasn't really paying close attention as new results were published and the last InstructGPT step makes such a big difference to how the tech is perceived. Actual knowledge doesn't change much but once it's housetrained, suddenly it's so much easier to interact with and use that it makes a step change in how accessible the tech is and how it's perceived.
I think I was more surprised by the joining of LLMs with generators and how well AI art works. It does feel like that happened fast. But, maybe I just wasn't paying attention again.
So I guess where we differ is that I don't take their surprise at face value. The direction was too clear, the gap between the threats they talk about in the abstract and the concrete proposals are too large and too lacking in obvious logical connection; it feels like motivated reasoning to me. I'm not entirely sure what's really going on and perhaps they are genuine in their concerns but if so it's hard to understand why they struggle so much to make a convincing case given they are certainly intellectually equipped to do so.
The two posts you linked are interesting and much better argued than the website this thread is about, so I'll reply to them directly.
Re: Statement on AI Risk
#957Earlier quoted context omitted.
> How can it not be obvious to you It isn't obvious to me. And I've yet to read something that spills out the obvious reasoning. I feel like everything I've read just spells out some contrived scenario, and then when folks push back explaining all the reasons that particular scenario wouldn't come to pass, the counter argument is just "but that's just one example!" without offering anything more convincing. Do you ha…
OK, which of the following propositions do you disagree with? 1. AIs have made rapid progress in approaching and often surpassing human abilities in many areas. 2. The fact that AIs have some inherent scalability, speed, cost, reliability and compliance advantages over humans means that many undesirable things that could previously not be done at all or at least not done at scale are becoming both feasible and cost-e…
Where we depart is point 4. Actually, both point 3 and 4 are things I agree with, but it's implied there's a logical link or progression between them and I don't think there is. The problem is the definitions of "outperform humans" and "smart".
Current AI can perform at superhuman levels in some respects, yes. Midjourney is extremely impressive when judged on speed and artistic skill. GPT-4 is extremely impressive whilst judged on its own terms, like breadth of knowledge. Things useful to end users, in other words. LLMs are deeply unimpressive judged on other aspects of human intelligence like long term memory, awareness of time and space, ability to learn continuously, willingness to commit to an opinion, ability to come up with interesting new ideas, hide thoughts and all that follows from that like being able to make long term plans, have agency and self-directed goals etc ... in all these areas it is weak. Yet, most people would incorporate most of them into their definition of smart.
Will all these problems be solved? Some will, surely, but for others it's not entirely clear how much demand there is. Boston Robotics was making amazing humanoid parkour bots for years yet the only one they seem able to actually sell is dog-like. Apparently the former aren't that useful. The unwillingness to commit to an opinion may be a fundamental trait of AI for as long as it's centralized, proprietary and the masses have to share a single model. The ability to come up with interesting new ideas and leaps of logic may or may not appear, it's too early to tell.
But between 3 and 4 you make a leap and assume that not only will all those areas be conquered very soon, but that the resulting AI will be unusually dangerous. The various social ills you describe don't worry me though. Bad governments will do bad things, same old, same old. I'm actually more worried about people using the existence of AI to deny true evidence rather than manufacture false evidence en-masse. The former is a lot of work and people are lazy. COVID showed that people's capacity for self-deception is unlimited, their willingness to deny the evidence of their own eyes is bottomless as long as they're told to do it by authority figures. You don't even need AI to be abused at all for someone to say, "ignore that evidence that we're clueless and corrupt, it was made by an AI!"
Then by point 6 we're on the usual trope of all public intellectuals, of assuming unending exponential growth in everything even when there's no evidence of that or reason to believe it. The self-improving AI idea is so far just a pipe dream. Whilst there are cases where AI gets used to improve AI via self-play, RLHF and so on, it's all very much still directed by humans and there's no sign that LLMs can self improve despite their otherwise impressive abilities. Indeed it's not even clear what self-improvement means in this case. It's a giant hole marked "??? profit!" at the heart of the argument. Neurosurgeons can't become superintelligences by repeatedly performing brain surgery on themselves. Why would AI be different?
Re: Statement on AI Risk
#958Earlier quoted context omitted.
There's a ton of stuff here and this isn't a climate thread, but briefly: 1. Yes I believe independents are more reliable than full time researchers because the latter are deeply conflicted and independents aren't. 2. They don't work for the oil industry. I've checked. That's propaganda designed to stop people listening. 3. There was in fact a very real pause, not simply due to statistical chance. Climatologists didn…
1. Noted. (I don't think this is a strong effect and think it is outweighed by the fact that the full-time researchers know and understand more.) 2. I said "funded by" rather than "employed by". For instance, consider James Taylor of the Heartland Institute, mentioned above. He doesn't "work for the oil industry" in the sense of being on the payroll of an oil company. (So far as I know, anyway.) But the Heartland Ins…
Here are two graphs from the same government agency (NASA), measuring the same thing (temperature), for the same time period, drawn twenty years apart. The data has been fundamentally altered such that the story it tells is different:
https://realclimatescience.com/wp-content/uploads/2019/08/NA...
The 2000-2015 pause is the same. You've been told that only "unorthodox" people were talking about it. Here's a brief history of the pause, as told by climatologists publishing in Nature.
https://www.nature.com/articles/nature12534
https://www.nature.com/articles/nature.2013.13832
In 2013 we read that "Despite the continued increase in atmospheric greenhouse gas concentrations, the annual-mean global temperature has not risen in the twenty-first century". The IPCC 2013 report is reported with the headline "Despite hiatus, climate change here to stay". Climatologists claim that maybe the missing heat has disappeared into the oceans where they can't find it.
https://www.nature.com/articles/nature.2013.13832
Two years later everything changes. "Climate-change ‘hiatus’ disappears with new data", there's a new version of history and the past 15 years are gone:
"That finding [that global warming actually did happen], which contradicts the 2013 report of the Intergovernmental Panel on Climate Change (IPCC), is based on an update of the global temperature records maintained by the US National Oceanic and Atmospheric Administration (NOAA)."
Climatologists have an interesting methodology - when observations don't fit the theory, they conclude the observations must be wrong and go looking for reasons to change them. Given a long enough search they always come up with something that sounds vaguely plausible, then they release a new version of the old data that creates new warming where previously there wasn't any. Although this isn't entirely secret they also don't tell people they're doing this, and the media certainly isn't going to tell anyone either.
And that's how it goes. You look at the edited graphs, remember people talking about a pause and think golly gosh, how badly informed those awful skeptics were. We have always been at war with Eastasia!
Re: Statement on AI Risk
#959Earlier quoted context omitted.
1. Noted. (I don't think this is a strong effect and think it is outweighed by the fact that the full-time researchers know and understand more.) 2. I said "funded by" rather than "employed by". For instance, consider James Taylor of the Heartland Institute, mentioned above. He doesn't "work for the oil industry" in the sense of being on the payroll of an oil company. (So far as I know, anyway.) But the Heartland Ins…
Whilst there's no need to reply, here is some information on the rewriting of history they engage in, as requested. Here are two graphs from the same government agency (NASA), measuring the same thing (temperature), for the same time period, drawn twenty years apart. The data has been fundamentally altered such that the story it tells is different: https://realclimatescience.com/wp-content/uploads/2019/08/NA... The 2…
(Also, that pair of graphs can't possibly be an example of changing historical data to get rid of a hiatus starting in 1998, for obvious reasons.)
I think you have misunderstood in multiple ways what I was saying about the "hiatus" starting in 1998.
Firstly, I was not claiming that only the "unorthodox" mentioned it. I was claiming that the "unorthodox" represented it as showing that global warming had stopped and the "orthodox" said otherwise. Your pair of Nature links are of "orthodox" climatologists saying things along the lines of "here is what we think may be the reason why the last few years haven't seen a short-term increase; the underlying trend is still upward": in other words, they are examples of the orthodox saying what I said they said.
Secondly, perhaps what I said about "signal" and "noise" gave the impression that I think, or think that "orthodox" climatologists thought, that the "noise" is measurement error. That's not what I meant at all, and I apologize for not being clearer. The point is that the temperature at any given place and time is a combination of lots of factors; some operate on a timescale of decades (global warming due to rising CO2 levels and all that), some on a timescale of multiple years (El Niño), some on much shorter timescales still ("random" variation because the atmosphere is a chaotic system). However accurately you measure, you're still seeing this sort of combination of things, and a time series lasting (say) 10 years will not necessarily reflect what's happening on longer timescales.
The 15 years starting in 1998, viewed in isolation, really do show a slower warming trend than the claimed long-term behaviour. There's nothing unreal about that, and so far as I know "orthodox" climatologists never said otherwise. What they said, and continue to say, and what I am saying, is that this sort of local counter-trend variation is perfectly normal, is exactly what you should expect to see even if global warming is proceeding exactly the way that orthodox climatologists say it is, and is not grounds for making claims that global warming has/had stopped or never been real in the first place.
The "everything changes" article you quote isn't saying what you're trying to make it out to be saying. (You can find the PDF here: https://www.researchgate.net/profile/Tr-Karl/publication/277... .) The authors have done two things. First, some corrections to historical data. (If you look at the graph at the end of the article you'll see that these corrections are pretty small. Their description of what they changed and why sounds perfectly reasonable to me; if you have good reason to think it's bogus other than the fact that you liked the old version better, do by all means share it.) Second, including more recent data. 2013 and, more so, 2014 were pretty warm years.
But what makes me think that the "unorthodox" were wrong to proclaim the end of global warming in the early 21st century isn't tiny adjustments in the data that make the difference between a +0.03 degC/decade trend between 1998 and 2012 and a +0.11 degC/decade trend between 2000 and 2014. It's the fact that after that period the short-term trend gets much faster, exactly as you would expect if the "hiatus" was simply the result of superimposing short-term fluctuations on a long-term trend that never went away.
I bet you are not 100% wrong about the tendency to adjust things to try to correct perceived anomalies. That's human nature, and while scientific practice has a bunch of safeguards to try to make it harder to do it would be surprising if they worked perfectly. But note that the sort of small local tweakage this might produce can't do much in the long term. Let's suppose those people writing in 2015 were completely wrong to make the adjustments they did, whether out of dishonesty or honest error (which maybe they were more inclined to overlook because the adjusted data looked more plausible to them). Then, yeah, they get a faster rate of warming between 1998 and 2014. But those same adjustments will produce a slower rate of warming between, say, 2014 and 2024 when someone comes to estimate that. And the rate of warming between 1998 and 2024 will scarcely be affected at all by tweaks to the numbers circa 2010.
Your last paragraph is (at least as far as I'm concerned) completely wrong, though. I think it was perfectly reasonable to say that the warming trend between 1998 and say 2012 was much slower than the alleged longer-term trend. What I think wasn't reasonable, and what I think has been refuted by later data, and what the "orthodox" climatologists said was wrong all along, was claiming that that short-term slower trend meant that the longer-term trend had gone away, or had never really been there in the first place. That was just statistical illiteracy, and Team Unorthodox were pretty keen on it, and that doesn't speak well for their competence and honesty.
Re: Statement on AI Risk
#960Earlier quoted context omitted.
Don't characterize the public as that stupid. The current risks of AI are startling clear to a layman. The extinction level even is more far fetched to a layman. You are the public and your viewpoint is aligned with the public. Nobody is thinking extinction level event.
Extinction is exactly what this submission is about. Here is the full text of the statement: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." By "extinction", the signatories mean extinction of the human species.