This seems like a step in the dangerous direction.
I am also concerned about existential threats from AI, but part of the problem is that I have no idea which research directions help and which ones hurt.
Meta-Transformer: A unified framework for multimodal learning
21–30 of 39 posts
Re: Meta-Transformer: A unified framework for multimodal learning
#22Yo dawg, we heard you like transformers so we put transformers on your transformers so you can train while you train. The spider web graph shows metatransformers performing worse to their counterparts in almost all fields. Is there a reason I should not believe that an expert model will always outperform a general purpose one, even if it's a metatransformer?
[dead]
Re: Meta-Transformer: A unified framework for multimodal learning
#23Yo dawg, we heard you like transformers so we put transformers on your transformers so you can train while you train. The spider web graph shows metatransformers performing worse to their counterparts in almost all fields. Is there a reason I should not believe that an expert model will always outperform a general purpose one, even if it's a metatransformer?
I'm pretty sure it's a relatively small model?
If you had the same quantity of text data as GPT-4 + comparable quantity of data for other domains, it could probably learn transferable skills across those domains.
But it would take a huge amount of processing power that is probably not attainable today
Re: Meta-Transformer: A unified framework for multimodal learning
#24Earlier quoted context omitted.
It’ll be ok. The technology for “dangerous” AI doesn’t actually exist. The near term risks we face from AI are constrained to the realms of spam and privacy. World ending super-bots are science fiction.
> It’ll be ok. The technology for “dangerous” AI doesn’t actually exist. Nobody's worried about the tech that exists. > The near term risks we face from AI are constrained to the realms of spam and privacy. Define "near term". > World ending super-bots are science fiction. Science fiction has become science fact before. Where's the knockdown argument that won't happen in this case?
[deleted]
Re: Meta-Transformer: A unified framework for multimodal learning
#25Earlier quoted context omitted.
> We're clearly at the transformer limits imho GPT-4 did not scale up substantially in depth, going from 175 b to 220 b per transformer.
Wouldn't making the model multimodal require scaling the models significantly? Or is the idea to keep the network the same size and trade off some of its nodes for image, video, etc. data? If so has anyone shown that doing so results in better overall performance? My lay-observation is that GPT-4 seems to be on the border of usability for most applications so if nothing is gained by simply changing the input data typ…
> Wouldn't making the model multimodal require scaling the models significantly?
Just width if that makes sense. Basically, you add another encoder model but you are not actually increasing the depth that much.
Re: Meta-Transformer: A unified framework for multimodal learning
#26Just few more steps like this, put it in a robot body, and Voilà , we have start of the first AI wars. How many centuries after this does the Butlerian Jihad start, lead by John Conner, of course?
Re: Meta-Transformer: A unified framework for multimodal learning
#27This seems like a step in the dangerous direction.
Why?
Because up till now many people that discount AI threats base that discount on a few assumptions like 'its just a parrot', 'it doesn't have any drives', 'it doesn't really understand', 'it isn't conscious', etc... ad-Infinium.
But the more different technology is plugged together to start resembling a brain, like a visual cortex, a speech center, motor controls, etc...
At some point the distinction between carbon based life and silicon becomes meaningless vanishes. All the arguments or proofs that humans are conscious would equally prove AI is conscious. Or that neither truly are. Proving an AI is not conscious would also prove humans aren't.
And of course, Terminators.
Re: Meta-Transformer: A unified framework for multimodal learning
#28Earlier quoted context omitted.
It’ll be ok. The technology for “dangerous” AI doesn’t actually exist. The near term risks we face from AI are constrained to the realms of spam and privacy. World ending super-bots are science fiction.
> It’ll be ok. The technology for “dangerous” AI doesn’t actually exist. Nobody's worried about the tech that exists. > The near term risks we face from AI are constrained to the realms of spam and privacy. Define "near term". > World ending super-bots are science fiction. Science fiction has become science fact before. Where's the knockdown argument that won't happen in this case?
Its not feasible to worry about the implications of every imaginary technology. Nuclear chain reactions were first theorized to exist a decade before the first bomb dropped. Should scientists have stopped exploring quantum mechanics in the 30s? Fear of the unknown shouldn’t be allowed to stop scientific progress.
We can deal with the implications of dangerous AI if and when it becomes a problem.
Re: Meta-Transformer: A unified framework for multimodal learning
#29This seems like a step in the dangerous direction.
It’ll be ok. The technology for “dangerous” AI doesn’t actually exist. The near term risks we face from AI are constrained to the realms of spam and privacy. World ending super-bots are science fiction.
Blind denial. No argument or evidence presented, merely bold statements made with the expectation they be taken without question.
Flying humans was science fiction 120 years ago. A single bomb able to destroy an entire city was science fiction 80 years ago. A machine that can complete more mathematical calculations in one minute than all human manual computation in history was science fiction 60 years ago. EUV photolithography capable of creating molecule-sized transistors was science fiction 30 years ago. A computer that can create visual art and talk to you in plain English was science fiction 2 years ago. A computer that can clone your voice and mannerisms was science fiction 1 year ago.
Science fiction has a way of becoming non-fiction, often within the span of a generation or less.
Re: Meta-Transformer: A unified framework for multimodal learning
#30This seems like a step in the dangerous direction.
It’ll be ok. The technology for “dangerous” AI doesn’t actually exist. The near term risks we face from AI are constrained to the realms of spam and privacy. World ending super-bots are science fiction.
>> "The technology for “dangerous” AI doesn’t actually exist"
What? Did you not see the Netflix documentary on AI for military use? They literally have AI's that can beat fighter pilots in dog fighting.
Just because it isn't walking around having coffee and chatting you up, doesn't mean it isn't already very advanced and deadly.