I'm chinapilled now. Anthropic is an evil, enemy of humanity company that cannot be allowed to succeed.
Yes. Which Chinese labs and models do you like the most?
Pacing the frontier
91–100 of 216 posts
Re: Pacing the frontier
#92The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…
"Many were increasingly of the opinion that they'd all made a big mistake in coming down from the trees in the first place. And some said that even the trees had been a bad move, and that no one should ever have left the oceans."
Re: Pacing the frontier
#93Earlier quoted context omitted.
And video games, democracy, novels, TV, cars, photography, cryptography, the Beatles… pick your favorite and read commentary / opinions of the time. Many thought that came with unique risks never seen before and must be stopped!
How many of those concerns came from the people building the technologies? John Carmack wasn't up in arms about how Doom was going to destroy society. Charles Dickens didn't think novels rotted the brain. Johannes Gutenberg wasn't concerned about the effects of a free press.
None of the previous concerns were true and nothing indicates current ones are. It’s extrapolation, hypothetical apocalyptic imagined scenarios and piles of assumptions on how the technology could evolve. What makes even less credible in the AI case is that there’s financial / power to be gained by those making the claims.
And to your point. Don Quixote, the first modern and greatest novel of all times, is about a guy that lost touch with reality from reading too many books
Re: Pacing the frontier
#94The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…
[deleted]
Re: Pacing the frontier
#95Earlier quoted context omitted.
This experiment has been run, and even the biological stuff is far fetched. I've written about this before, but the issue is that these people have worked with computers their entire lives, and they keep projecting outwards from there. The 20th century was dominated by mad scientists (mostly Teller) and generals (bombs away LeMay), but one thing that I respect about them is that they didn't just sit and guess about p…
Thank you for taking this seriously enough to write this, and anyone else likewise. But this is attacking a strawman, amateur bioterrorists. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening. Being at risk of killing yours…
But this seems like it's attacking a strawman. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening.
I want to ask you a question, do you feel like that this is the problem being solved by the current "safety features?"
Let me rephrase my question, do Anthropic and OpenAI implement the same "guardrails" and "safety features" for the NSA? Does the Mythos NSA Edition™ have these restrictions? What about the one that's running as a part of Maven?
How does stopping me from asking about rabbit sex protect you from nut jobs with a security clearance using these systems to go off and make weapons?
Because this has happened before. The only successful bioweapons attack in US history was done by a guy who worked at Fort Detrick. https://en.wikipedia.org/wiki/2001_anthrax_attacks
The only private organization successful enough to make biological and chemical weapons is Aum Shinrikyo and they had university affiliation and nearly a billion dollars in the 1990s.
The guy who led Aum's bioweapons program was
Seiichi Endo was born in 1961 and attended Obihiro university of Agricultural and Veterinary Medicine. After graduating, he became a student at the Kyoto university medical school research department focusing on AIDS-related gene research at the viral research center. He joined Aum in 1987, became a monk in 1988 and later led Aum’s biological weapons program.
Among some of Japan's "best and brightest" who joined the cult included a former researcher of the National Space Development Agency of Japan, an expert on chemical weapons who majored in organic physics at Tsukuba University, a researcher who studied elementary particles, a reporter with a major Japanese newspaper, a physicist from Osaka University, a cardiac specialist, and an organic chemist, to name a few.
and,
Through its network, Aum Shinrikyo successfully recruited over 300 scientists and engineers—including employees at the Kurchatov Institute, the premier nuclear facility in Russia—who were either attracted to the apocalyptic ideology or lured with financial incentives.50 Aum Shinrikyo’s Russian contacts enabled group members to access black market materials and hardware.
In response, the cult constructed the $30 million USD Satyan 7 facility which was equipped with three laboratories, a computer control center, and five reactors, all of which was made from corrosion-resistant Hastelloy, ideal for the production of chemical weapons
It's because they'd recruited members of the military, bribed local policemen and politicians, and used their sway to silence critics (or kill them).
How does stopping me from asking Claude about rabbit sex stop people like them?
From where I'm standing, Anthropic and OpenAI would have sold Aum a subscription.
The Fort Detrick guy would have access via the US Government and Aum had university affiliation. I am yet to read a serious proposal that actually deals with these risks and the other real risks of this technology.
Re: Pacing the frontier
#96Earlier quoted context omitted.
i think part of the problem, though, is that we've already hit many of the warning signs that AGI is coming, and people have kept moving the goalposts. if you were to tell someone from ten years that there's an ai that has passed the turing test and has made novel mathematical discoveries, i think many would say AGI is already here.
You're right, in that we would have said that. I wouldn't frame it as moving the goalposts in a bad way, though. We understand more now than we did then, so we have a better idea of what the actual goal is, and moving the goalposts to frame that goal is more rational than insisting the goalposts stay where they were ten years ago even though the entire pitch has changed since then. I'm straining the analogy, obviousl…
Moving the goalposts, so to speak, is how science works! We must of course update the things we think based on an improved understanding of how the world works. Only the deeply incurious could consistently demand that one adhere dogmatically to a prior way of understanding the world in the face of changing knowledge. It's important not to treat science as a game to be won (of which 'goalposts' are evocative), but as the collaborative effort that it is.
Note well that I have not said LLMs are not useful, only that no amount of interaction with them has convinced me they remotely approach general intelligence. Their actual function of heuristically regurgitating all information ever known to mankind remains extremely useful in lots of domains.
And by the way, it's not obvious an LLM could fool me (or the average person) over a sufficiently long period of time, exactly because they are heuristic machines. That they lack thought guided by underlying cognitive models seems to always leak out, in the end. I suspect pass rates by LLMs on long-range Turing tests (if there have been any conducted) might drop as people become acclimatized to them.
Re: Pacing the frontier
#97Earlier quoted context omitted.
That's an application of the "Tragedy of the commons": One person stopping to push the frontier just means that this person will lose their job, the frontier will be pushed by the others. The solution is to push for a binding rule instead that will make everyone slow down in the racing towards artificial superintelligence.
And this is how we absolve people of all personal responsibility. Such companies would not pay people that well if they were so fungible. There is no such possible binding rule either, a point that is so obvious that any smart person claiming otherwise is clearly up to something.
Hoping everyone chooses to do the right thing is not an effective strategy for mitigating bad things.
Re: Pacing the frontier
#98This. Will. Never. Work. This research game is a zero-trust iterated game. Any defection carries the benefit of possibly being permanently valuable. There will always be defections. OpenAI defected WHILE THEY WERE pushing to slow. The cost to the US of deciding to slow is so massively harmful that it is wrong to even put this idea out in the public zeitgeist; it may somehow become a thing that US legislators believe…
Some reason this kind of argument always by default assumes the US will be the good guys.
Re: Pacing the frontier
#99The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…
If that’s truly the case, why aren’t they offering to be absorbed by national labs. That’s what we do for dangerous biological and nuclear work.
Oh wait they want options and shares and to get rich.
And there’s no natural moat. Which Federal regulation would bring.
That’s why people think it’s disingenuous.
Re: Pacing the frontier
#100Can someone help clarify what it means, concretely, to make binding agreements to proceed "safely and slower" in the context of AI research (whether or not at frontier labs)? Are there specific policies and actionable guides being proposed?