I support direction of AI control, but I think focusing on size/compute isn't productive at all.
Use cases are more important: let it control browser on itself, let execute arbitrary code etc. Depending on capabilities of next GPT versions, opening GPT API could be risky.
Pause Giant AI Experiments: An Open Letter
151–160 of 1001 posts
Re: Pause Giant AI Experiments: An Open Letter
#152Earlier quoted context omitted.
Is anyone even trying to develop an AGI by the way? I mean a language model is neat, may even pick up some emergent features, but in the end it has no real identity. It's basically a memoryless computer doing improv where it tries to pick continue a conversation someone else had.
It used to be that only a few academics were actually working on AGI, and even they had to pretend they were working on narrow AI to get funding. But now, plenty of big labs are. For example, DeepMind's About page states: > Our long term aim is to solve intelligence, developing more general and capable problem-solving systems, known as artificial general intelligence (AGI). In the long term, I still think DeepMind wi…
Agreed. DeepMind and Hassabis are on the right track—-a society of modules with many layers of recursive self attentional gating. LMM is one big part though.
Re: Pause Giant AI Experiments: An Open Letter
#153It's too late. Anyone who pauses will simply be left behind. This is especially salient if you think US domination of tech is better for the world than Chinese domination -- if the US pauses, China certainly won't. (And yes, US domination or Chinese domination of tech really are the only realistic outcomes.)
Got to agree—unfortunately.
Re: Pause Giant AI Experiments: An Open Letter
#154Earlier quoted context omitted.
This is such an interesting use case. Rather can compressing existing information, LLMs "model" it, so they can recreate what they've learned. For all Elon's concerns, these might be a great way of seeding the universe with human knowledge, one planetary colony at a time...
That would be an amazing sci-fi novel, if the concept hasn’t been done yet.
Since you asked... https://gist.github.com/jessejohnson/3fdd6cfe614eb036532bf4d...
(Disclosure: AI written short story)
Re: Pause Giant AI Experiments: An Open Letter
#155For a long time, "AI alignment" was a purely theoretical field, making very slow progress of questionable relevance, due to lack of anything interesting to experiment on. Now, we have things to experiment on, and the field is exploding, and we're finally learning things about how to align these systems. But not fast enough. I really don't want to overstate the capabilities of current-generation AI systems; they're no…
The problem is, can alignment occur before the relevant capabilities have been developed? LLMs, for example, (although very good at impersonating and talking to humans and having good world models) are particularly poor at structured reasoning and planning which are the capabilities that will actually be so dangerous. I don't believe superintelligence will be a LLM with chain-of-thought reasoning. If it's a different architecture then once again a lot of alignment work won't be relevant.
Re: Pause Giant AI Experiments: An Open Letter
#156Stopping now would be extremely dangerous and borderline stupid.
If you stop now, you're just left behind, because there's no way everyone will stop.
At this point the only logical course of action in an adversarial situation is to double down and keep researching, otherwise some other country or culture with different (and possibly worse) values ends up dominating the technology and you're left behind in the dust.
The genie is out of the bottle, there's not putting it back in.
Re: Pause Giant AI Experiments: An Open Letter
#157I understand that some people find this a hopeless gesture. But I think anybody who thinks "losing jobs" or "China" is a bad outcome from AI is hopelessly lost. The upside of AI if done right - nobody ever having to work again The downside of AI if done wrong - end of humanity It doesn't take a rocket scientist to determine we've made AIs go from thinking like an ant to thinking like a 6-year-old in a few years. Anyb…
Stopping it's not a hopeless gesture, it's dangerous as fuck.
If you stop research but all your adversaries don't, you're left behind.
Let's assume that all western societies stop, but let's say, China doesn't or Russia, and they keep advancing the technology.
If the tech brings efficiencies in play, and you don't have access to them, you're in a really bad place, maybe even worse off that if you had continued and risked the status quo because you were afraid.
The genie is out of the bottle at this point, there's not putting it back in.
Re: Pause Giant AI Experiments: An Open Letter
#158For a long time, "AI alignment" was a purely theoretical field, making very slow progress of questionable relevance, due to lack of anything interesting to experiment on. Now, we have things to experiment on, and the field is exploding, and we're finally learning things about how to align these systems. But not fast enough. I really don't want to overstate the capabilities of current-generation AI systems; they're no…
>... the risk of a superintelligence taking over the world no longer looks distant and abstract.
Can we please stop floating this as a threat? This is the more science-fiction than reality at this point and it does a great disservice to humanity. The more we keep pushing the idea that AI is the threat and not the people controlling it the less we will be focused on mitigating global risk.
It is far more likely that someone else will leverage an AI to attempt to expand their influence or dominion. Putin has essentially already stated views on this matter and we should assume groups within all adequately advanced nations will be working toward this end either independently or cooperatively.
We are more than likely in an arms race now.
Re: Pause Giant AI Experiments: An Open Letter
#159For a long time, "AI alignment" was a purely theoretical field, making very slow progress of questionable relevance, due to lack of anything interesting to experiment on. Now, we have things to experiment on, and the field is exploding, and we're finally learning things about how to align these systems. But not fast enough. I really don't want to overstate the capabilities of current-generation AI systems; they're no…
The problem is, can alignment occur before the relevant capabilities have been developed? LLMs, for example, (although very good at impersonating and talking to humans and having good world models) are particularly poor at structured reasoning and planning which are the capabilities that will actually be so dangerous. I don't believe superintelligence will be a LLM with chain-of-thought reasoning. If it's a different…
Yes, many angles on the alignment problem can be studied now, and have started making good progress recently. Some things will turn out in retrospect to not have been relevant, due to architectural shifts, but not everything. Some things are specific to LLMs; some things are specific to transformers but not to language-model transformers; some things are conceptual and likely to still apply to quite-different systems; and some things are just field-building and not specific to any architecture at all.
Eg in mechanistic interpretability, there are a lot of findings on LLMs that turn out to generalize across a wider set of NN architectures. Eg https://transformer-circuits.pub/2022/solu/index.html is something that couldn't be done without access to LLMs, but which looks likely to generalize into future architectures.
Re: Pause Giant AI Experiments: An Open Letter
#160For a long time, "AI alignment" was a purely theoretical field, making very slow progress of questionable relevance, due to lack of anything interesting to experiment on. Now, we have things to experiment on, and the field is exploding, and we're finally learning things about how to align these systems. But not fast enough. I really don't want to overstate the capabilities of current-generation AI systems; they're no…
> ... the risk of a superintelligence taking over the world no longer looks distant and abstract. Can we please stop floating this as a threat? This is the more science-fiction than reality at this point and it does a great disservice to humanity. The more we keep pushing the idea that AI is the threat and not the people controlling it the less we will be focused on mitigating global risk. It is far more likely that…
So once again, humans are the dangerous part, clearly, if we didn't have destructive tendencies in our psyche that we're using to train these models, we wouldn't build things that would be interested in destruction.
Interesting.
I don't think we're as intelligent as we believe we are which I doubt we will ever actually build a super intelligence, we're too stupid. Even something 10x smarter than us may actually be quite "stupid".