41–50 of 93 posts
Earlier quoted context omitted.
The model isn't available at all?
It is available in the sense that it is accessible. The weights are not available for download of course, but the OP wanted to "play around" with it, for which only access is required. There is no other accessible foundation model that can compete with GPT-3.5.
Earlier quoted context omitted.
The most powerful available foundation model is code-davinci-002, a.k.a. GPT-3.5. It's only available on Azure since OpenAI removed it from their own Playground and API for some reason.
Maybe you mean gpt-3.5-turbo or text-davinci-003? Or GPT-4 (technically in beta so not fully available to everyone)?
> trust_remote_code=True This is a hard no from me, anyone know why this is so common in models from China? I'm not getting into conspiracies or anything here, but I've seen it in quite a few others from there. I wouldn't run a model with this requirement from anyone else for that matter.
That's because the model architecture hasn't been added to huggingface/transformers yet, because it literally was just published today. >>> from transformers import AutoTokenizer, AutoModel >>> model = AutoModel.from_pretrained("internlm/internlm-chat-7b", trust_remote_code=True, device='cuda') Here, the "trust_remote_code=True" means "download the model code from huggingface repo 'internlm/internlm-chat-7b'", along…
This makes sense in a way given the API of typical ML libraries. But there is no fundamental reason this needs to be the case.
Or, more correctly stated: model == code for sure, but said code need not have any rights to perform side effects. For some reason e.g. TensorFlow has stuff like tf.io.write_file [1] (is that actually an operation you can put in a model???), but one could easily imagine a more appropriate domain-specific model language that your code is compiled to, that can by design not perform any IO. Imagine that a model you distribute is not random Python code that may or may not run a model, but instead the model itself, i.e. the graph encoded in that domain-specific language.
Then downloading a random model from some random untrusted place is no different from downloading some random data from some untrusted place: you're going to execute the model, which may DOS you, but nothing much else will happen.
Unfortunately the ML world is too stuck in the imperative mindset for this (IMO more sensible) way of doing things. :)
[1]: https://www.tensorflow.org/api_docs/python/tf/io/write_file
Earlier quoted context omitted.
That's because the model architecture hasn't been added to huggingface/transformers yet, because it literally was just published today. >>> from transformers import AutoTokenizer, AutoModel >>> model = AutoModel.from_pretrained("internlm/internlm-chat-7b", trust_remote_code=True, device='cuda') Here, the "trust_remote_code=True" means "download the model code from huggingface repo 'internlm/internlm-chat-7b'", along…
> newcomers may not realize that model == code This makes sense in a way given the API of typical ML libraries. But there is no fundamental reason this needs to be the case. Or, more correctly stated: model == code for sure, but said code need not have any rights to perform side effects. For some reason e.g. TensorFlow has stuff like tf.io.write_file [1] (is that actually an operation you can put in a model???), but…
It's highly unlikely that the market for running these models like an appliance securely in an untrusted context will ever manifest. It's just too much of a niche, as it would also reduce their extensibility/usability significantly
Earlier quoted context omitted.
Less excited that you can't freely take work from academics to resell it in a shitty SaaS company ?
You do understand that academics are usually funded by taxpayers? Obviously, not by me, as I don't pay taxes in China, but it's not like academics are doing this work for free. Society pays them for their work so that it can benefit from its results.
Earlier quoted context omitted.
Less excited that you can't freely take work from academics to resell it in a shitty SaaS company ?
You do understand that academics are usually funded by taxpayers? Obviously, not by me, as I don't pay taxes in China, but it's not like academics are doing this work for free. Society pays them for their work so that it can benefit from its results.
Earlier quoted context omitted.
Maybe you mean gpt-3.5-turbo or text-davinci-003? Or GPT-4 (technically in beta so not fully available to everyone)?
No, those are all fine-tuned models which are "nerfed" in the terminology of the OP. I mean code-davinci-002, the GPT-3.5 base model.
Earlier quoted context omitted.
No, those are all fine-tuned models which are "nerfed" in the terminology of the OP. I mean code-davinci-002, the GPT-3.5 base model.
Is that what nerfed means? I usually see "nerfed" used in a way that means that it will refuse to answer certain topics. "I can't answer that as it would violate copyright" and such.