InternLM – new open source 7B LLM • Hacker News Board

Re: InternLM – new open source 7B LLM

#71

Post by czDRZ-akk » Thu, Jul 06, 2023, 4:15 PM UTC

airgapstopgap wrote: post #11

Note that this is apparently a 7B version of a 104B model trained with the intention of competing with OpenAI offerings on the Chinese market [1]. There is a number of those projects: Baichuan, ChatGLM2, InternLM and some more iirc, and they all have small-scale opensource versions. For what it's worth, I've tried out ChatGLM2-6B and Baichuan converted to LLaMA (the architecture is literally identical in that case).…

Chinese regulation around generative AI isn’t yet formalized, including provisions for censorship. The Cyberspace Administration of China published a set of draft measures[0] for public comment, but it doesn’t seem like a revised version has been released.

The draft indicates that there will be some level of censorship, but it’s unclear what the scope will be. This analysis[1] suggests that generative AI for research purposes could be exempted (section 1). The same analysis points out that there are other government bodies at play that are more focused on advancing AI as an industry within China.

It does seem likely that there will be some kind of censorship carve-out for AI research, whereas companies offering generative AI products to the public will need to self-censor to avoid fines and/or prosecution.

[0] https://digichina.stanford.edu/work/translation-measures-for...

[1] https://fpf.org/blog/unveiling-chinas-generative-ai-regulati...

Re: InternLM – new open source 7B LLM

#72

Post by joelthelion » Thu, Jul 06, 2023, 5:02 PM UTC

What kind of hardware do you need to run a model like this? Do you need an A100, or can something smaller suffice?

What about for fine tuning? Are hardware requirements higher?

Re: InternLM – new open source 7B LLM

#73

Post by freedomben » Thu, Jul 06, 2023, 5:09 PM UTC

ohgodplsno wrote: post #51

version_five wrote: post #36

Earlier quoted context omitted.

So you don't use open source software? You should try it, there's a great ecosystem of free software, including lots written by academics who are happy to have their work add value to industry

I do. And I also don't use open-source software with a commercial license for my job, because I respect the wishes of the author. It doesn't make the projects, tools and libraries any less good. OP is just looking to quickly cash in to something he didn't put an ounce of effort in. AGPL & Dual-licensing are the way forward, because of leeches.

I neither agree nor disagree with your position (still thinking about it), but I do think that's right uncharitable mind-reading you've done of GP. They never said anything about "looking to quickly cash in to something he didn't put an ounce of effort in."

Re: InternLM – new open source 7B LLM

#74

Post by echelon » Thu, Jul 06, 2023, 5:19 PM UTC

version_five wrote: post #37

exizt88 wrote: post #13

> The code in this repository is open-source under the Apache-2.0 license. The InternLM weights are fully open for academic research and also allow commercial use with written permission from the official team. For inquiries about commercial licenses and collaborations, please contact internlm@pjlab.org.cn. This makes me much less excited about this model.

Agreed, this basically moves it to the "don't bother" pile. There are already the llama variants with non-commercial licenses, and open-llama as an open source model (I'm thinking in the 7B space specifically). This would have to be pretty friggin compelling to spend any time on.

Do they even have the legal justification of saying how you can or cannot use the weights? It could be ruled that weights are uncopyrightable. I think we as a community should advocate for that.

If you train on data you don't own, the results (weights, unmodified outputs) should be public domain. When people create novel works on top (SaaS tools, music, films), then those human combinations should hold copyright. Not the model weights.

If you can prove you own all of the inputs into training, then perhaps it's another story. But that could also be dangerous and allow for data cartels to own the future.

Re: InternLM – new open source 7B LLM

#75

Post by [deleted] » Thu, Jul 06, 2023, 7:10 PM UTC

huggingmouth wrote: post #57

exizt88 wrote: post #23

Earlier quoted context omitted.

The model code is Apache 2.0, the weights are proprietary.

That's not their huggingface repo says: https://huggingface.co/internlm/internlm-7b The current release on huggingface is available under plain apache 2.0

[deleted]

Re: InternLM – new open source 7B LLM

#76

Post by timmclean » Thu, Jul 06, 2023, 7:18 PM UTC

Saving you a click: despite what the repo title might suggest, while the code is open source, the model weights cannot be used commercially without permission.

> The code in this repository is open-source under the Apache-2.0 license. The InternLM weights are fully open for academic research and also allow commercial use with written permission from the official team. For inquiries about commercial licenses and collaborations, please contact internlm@pjlab.org.cn.

https://github.com/InternLM/InternLM#open-source-license

Re: InternLM – new open source 7B LLM

#77

Post by cubefox » Thu, Jul 06, 2023, 8:13 PM UTC

brucethemoose2 wrote: post #63

cubefox wrote: post #41

Earlier quoted context omitted.

Why are you guys downvoting me?

Because GPT 3.5 not very good compared to LLaMA 65b or even 33b finetunes, from my testing. Also because 3.5 is not really available?

Have you actually tested code-davinci-002?

Re: InternLM – new open source 7B LLM

#78

Post by cubefox » Thu, Jul 06, 2023, 8:14 PM UTC

seanhunter wrote: post #65

cubefox wrote: post #43

Earlier quoted context omitted.

No, those are all fine-tuned models which are "nerfed" in the terminology of the OP. I mean code-davinci-002, the GPT-3.5 base model.

code-davinci models are finetuned on code so I don't think that's what the OP wants. For reference the family tree is here https://platform.openai.com/docs/model-index-for-researchers

As the website you linked says, code-davinci-002 is not fine-tuned. It is the GPT-3.5 base model.

Re: InternLM – new open source 7B LLM

#79

Post by cubefox » Thu, Jul 06, 2023, 8:16 PM UTC

alach11 wrote: post #69

cubefox wrote: post #43

Earlier quoted context omitted.

No, those are all fine-tuned models which are "nerfed" in the terminology of the OP. I mean code-davinci-002, the GPT-3.5 base model.

> I mean... the GPT-3.5 base model That would be text-davinci-003, I believe.

No, text-davinvi-003 is fine-tuned. The base model is code-davinci-002. See https://platform.openai.com/docs/model-index-for-researchers

Re: InternLM – new open source 7B LLM

#80

Post by ForOldHack » Thu, Jul 06, 2023, 8:53 PM UTC

brucethemoose2 wrote: post #64

wilonth wrote: post #53

Earlier quoted context omitted.

7B params would take 14gb of gpu RAM at fp16 precision. So it would be able to run on 16gb GPUs with 2gb to spare for other small things.

But in practice, no one is running inference at FP16. int8 is more like the bare minimum.

I have an 8GB, and I am considering two more 8GB, it should I get a single 16GB? The 8GB card was donated, and we need some pipelining... I have 10~15 2GB quadro cards... Apparently useless.