Does anyone else think closed AI is turning into it's most weirdest forms and becoming a trend?
OPT: Open Pre-trained Transformer Language Models
231–240 of 242 posts
Re: OPT: Open Pre-trained Transformer Language Models
#232Earlier quoted context omitted.
> so 30 12gb gpus - a 3090 will cost around $1800 3090 has 24Gb, thus 15 GPUs X $1800 = $27,000 in GPUs
Can 3090 GPUs share their memory with one another to fit such a large model? Or is the enterprise grade hardware required?
Yes, two 3090s ($1.7k each) can be connected via NVlink with common 48Gb of memory pool.
Two RTX A6000 ($4.5k each) can form 96Gb memory pool.
Re: OPT: Open Pre-trained Transformer Language Models
#233What type of hardware would you need to run it?
A cluster of many $8000+ gpus. You're looking at around 350GB of vram, so 30 12gb gpus - a 3090 will cost around $1800, so $54k on the gpus, probably another $15k in power, cooling, and infrastructure, $5k in network, and probably another $20k in other costs to bootstrap it. Or wait 10 years, if gpu capacity scales with Moore's law, consumer hardware should be able to run a ~400GB model locally.
One could use $4.5k RTX A6000 48Gb instead.
They can be joined in pairs of 96Gb common memory pool with NVlink.
That’s 7x$4.5=$31.5k in GPUs to get 336Gb of memory.
Or 8x$4.5=$36k in GPUs to get 384Gb of memory.
Add say $3k per GPU pair for surrounding computer (MB,CPU,RAM,PSU) 4x$3k=$12k.
$48k total budget.
Re: OPT: Open Pre-trained Transformer Language Models
#234Earlier quoted context omitted.
Or slightly randomly modify all the parameters on the copy you distribute, then it will be a match for nobody.
You compare all three and average the variance of each value. So the more copies the better.
...or just steal it so that even if it can be traced, it's not your problem.
Re: OPT: Open Pre-trained Transformer Language Models
#235Just curious, will I be able to use it using my Nvidia card with 10GB of memory? Does it require multiple graphic cards?
As the model weights (even quantized) would be several hundred GBs, it’s unlikely, unless special inference code is written that loads and processes only a small subset of weights and calculations at a time. But running it that way would be painfully slow.
The code is already there: DeepSpeed
Re: OPT: Open Pre-trained Transformer Language Models
#236A quick summary of the Limitations section: - "OPT-175B does not work well with declarative instructions or point-blank interrogatives." - "OPT-175B also tends to be repetitive and can easily get stuck in a loop. While sampling can reduce the incidence rate of repetitive behavior (Holtzman et al., 2020), we anecdotally found it did not eliminate it entirely when only one generation is sampled." - "We also find OPT-17…
> OPT-175B has a high propensity to generate toxic language and reinforce harmful stereotypes So they trained it on Facebook comments?
We trained on Reddit comments and HackerNews comments.
Re: OPT: Open Pre-trained Transformer Language Models
#237Earlier quoted context omitted.
A 175 billion parameter model might be a couple hundred gigs on disk. The file is probably just too big for GitHub/other standard FB services.
They could just torrent.
They could, and their might be a torrent in the future - but torrents lose tracking info. I'm sure the researchers want to know who is downloading their models even if they don't care who it is.
Re: OPT: Open Pre-trained Transformer Language Models
#238Re: OPT: Open Pre-trained Transformer Language Models
#239Re: OPT: Open Pre-trained Transformer Language Models
#240Earlier quoted context omitted.
Slightly modify a million random weights by changing the least significant bit up or down.
Compare three copies.
OK, this is a fun game. I think your counterattack assumes I'm picking these million weights uniformly randomly among the 175 billion. I modify my original answer: s/a million/half the weights in a deterministic subset of 2 million weights/
Select the deterministic subset by just hashing some identifier for each weight.
For any reasonable number of copies, there's a pretty unique subset between all your copies sharing a large amount of bits flipped in the same direction among this subset.