31–40 of 51 posts
There are non-OpenAI models based on the same GPT paper as the OpenAI GPT-series, e.g., GPT-NeoX [0], GPT-J, etc., that are actually Open Source, unlike OpenAI which is “open” only in the sense of “we might let you use it, either as a free preview or a paid service”. You probably won't be able to run (or especially train) them on typical desktops, though. [0] https://www.eleuther.ai/projects/gpt-neox/
I'm pretty sure the GPT model is huge and does not fit on any conventional GPU. Even if they open-sourced the weights, I don't think most people would be running it at home. Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able…
Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.
There are non-OpenAI models based on the same GPT paper as the OpenAI GPT-series, e.g., GPT-NeoX [0], GPT-J, etc., that are actually Open Source, unlike OpenAI which is “open” only in the sense of “we might let you use it, either as a free preview or a paid service”. You probably won't be able to run (or especially train) them on typical desktops, though. [0] https://www.eleuther.ai/projects/gpt-neox/
Are they good?
To replace ChatGPT, no they are not good enough.
Earlier quoted context omitted.
Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.
Silly question: how does OpenAI host/serve it?
Luckily, no. Otherwise you (others) could hack the safeties and ask it how to cheaply kill a lot of people or so .. better not make bad people too intelligent! (Obviously not talking about you)
It's not magic, it doesn't know anything that isn't easily findable public information.
"in Google sheets, I have dates in the first column. How would I make the second column indicate if daylight savings time is active on that date?"
ChatGPT will confidently provide non working suggestion after non working suggestion, but ultimately can't make this work.
There are non-OpenAI models based on the same GPT paper as the OpenAI GPT-series, e.g., GPT-NeoX [0], GPT-J, etc., that are actually Open Source, unlike OpenAI which is “open” only in the sense of “we might let you use it, either as a free preview or a paid service”. You probably won't be able to run (or especially train) them on typical desktops, though. [0] https://www.eleuther.ai/projects/gpt-neox/
Are they good?
Since my other account is shadow banned for some unexplained reason, I just wanted to mention the petal project. It's an attempt to bittorrent style distribute the load of running these large models. Good luck!
There are non-OpenAI models based on the same GPT paper as the OpenAI GPT-series, e.g., GPT-NeoX [0], GPT-J, etc., that are actually Open Source, unlike OpenAI which is “open” only in the sense of “we might let you use it, either as a free preview or a paid service”. You probably won't be able to run (or especially train) them on typical desktops, though. [0] https://www.eleuther.ai/projects/gpt-neox/
Are they good?
GPT-NeoX apparently compares favorably with GPT-3 on some measures, but ChatGPT is a 175 billion parameter GPT-3 model, and the big pretrained GPT-NeoX model available is a 20 billion parameter model. Could you rival ChatGPT with the right settings and training set and sufficient hardware for training? Well, if you want to try, you can with GPT-NeoX, and you can chosr whether or how to filter the output. With OpenAI’s models, you get what they give you, on the terms they are willing you give you access to it, with filters that exist tonprotect OpenAI’s image and liability.
OpenAI is 'open' in the name only, so no. I don't think they have any plans of opening full access to the public either, considering that their previous model (that ChatGPT builds upon) was sold to Microsoft for exclusive use: https://en.wikipedia.org/wiki/GPT-3
It's really obnoxious for non-open companies to include "open" in the name. Stolen valour, basically.
Though, I too, would rather be able to run the model myself.