Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

InternLM – new open source 7B LLM

github.com

Post reply on HN
Print view View original

81–90 of 93 posts

Re: InternLM – new open source 7B LLM

#81

Post by cubefox »

seanhunter wrote: post #61
cubefox wrote: post #24

Earlier quoted context omitted.

The most powerful available foundation model is code-davinci-002, a.k.a. GPT-3.5. It's only available on Azure since OpenAI removed it from their own Playground and API for some reason.

All 3 text-davinci models are available on openAI's api. including 3 (which is the GPT-3.5 gen). Code-davinci-002 is a code-tuned model, You can see a nice visual summary of the relationships between the openAI models at https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr... Or the official source is https://platform.openai.com/docs/model-index-for-researchers

> All 3 text-davinci models are available on openAI's api.

That's irrelevant because these are all fine-tuned.

> Code-davinci-002 is a code-tuned model

No, "code-tuned" isn't even a thing. It is a foundation model, which consists purely of pretreating. No fine-tuning is involved.

> Or the official source is

The official source says exactly what I just said.

Re: InternLM – new open source 7B LLM

#82

Post by b1n »

There is a great opportunity for totalitarian and authoritarian regimes (China and UAE so far) to create commercially usable and free LLMs that work significantly better than alternatives (backed by large amounts of government money).

Over time, as they get used in more and more products, these LLMs can become more 'aligned' to these regimes way of thinking.

There are no Chinese companies that are not part of the Chinese government.

This is a new kind of cultural soft power.

Re: InternLM – new open source 7B LLM

#83

Post by airgapstopgap »

brucethemoose2 wrote: post #59
airgapstopgap wrote: post #11

Note that this is apparently a 7B version of a 104B model trained with the intention of competing with OpenAI offerings on the Chinese market [1]. There is a number of those projects: Baichuan, ChatGLM2, InternLM and some more iirc, and they all have small-scale opensource versions. For what it's worth, I've tried out ChatGLM2-6B and Baichuan converted to LLaMA (the architecture is literally identical in that case).…

> Surprisingly, they do not appear censored in any particularly "Chinese" political direction, but they share sensibilities of ChatGPT and Claude. Perhaps they used GPT4 responses for the instruct finetuning, as many LLaMA finetunes do? The paper doesn't say where they got the data from, other than "The pre-trained language model is further fine-tuned, following the mainstream procedure as in InstructGPT." (Also, I d…

I believe it's more like they used Anthropic human preference data [1] or similar, and accordingly Anthropic/progressive American notion of honest-helpful-harmless behavior. Thus I've seen models misgeneralize towards prudish finger-wagging. For example they parse badwords like "beat", "abuse", "steal" in morally neutral contexts ("beat a benchmark" or something) as signifiers of substantial transgression and spiral into telling me how, as language models, they insist it's never okay to etc. etc. This attitude was strikingly reminiscent of American models, even though other failure modes – like hallucinations – don't seem so similar.

Papers like Tulu [2] suggest that LLaMA-65b is indeed an appropriate baseline, given reasonable prompting. Instruct datasets only convey a flavor of responses, and for a strong foundation model that can infer the intended flavor on its own, naive finetuning seems to be detrimental. GPT-4 was much more powerful prior to having been finetuned, if reports of early witnesses and researchers are to be believed.

1. https://huggingface.co/datasets/Anthropic/hh-rlhf

2. https://arxiv.org/abs/2306.04751

Re: InternLM – new open source 7B LLM

#84

Post by brucethemoose2 »

ForOldHack wrote: post #80
brucethemoose2 wrote: post #64

Earlier quoted context omitted.

But in practice, no one is running inference at FP16. int8 is more like the bare minimum.

I have an 8GB, and I am considering two more 8GB, it should I get a single 16GB? The 8GB card was donated, and we need some pipelining... I have 10~15 2GB quadro cards... Apparently useless.

I mean... It depends?

You are just trying to host a llama server?

Matching the VRAM doesn't necessarily matter, get the most you can afford on a single card. Splitting beyond 2 cards doesn't work well at the moment.

Getting a non Nvidia card is a problem for certain backends (like exLLaMA) but fine for llama.cpp in the near future.

AFAIK most backends are not pipelined, the load jumps sequentially from one GPU to the next.

Re: InternLM – new open source 7B LLM

#85

Post by heartbeats »

b1n wrote: post #82

There is a great opportunity for totalitarian and authoritarian regimes (China and UAE so far) to create commercially usable and free LLMs that work significantly better than alternatives (backed by large amounts of government money). Over time, as they get used in more and more products, these LLMs can become more 'aligned' to these regimes way of thinking. There are no Chinese companies that are not part of the Chi…

Aligned how? If you download the code, it won't change under your feet.

Re: InternLM – new open source 7B LLM

#86

Post by ASalazarMX »

heartbeats wrote: post #85
b1n wrote: post #82

There is a great opportunity for totalitarian and authoritarian regimes (China and UAE so far) to create commercially usable and free LLMs that work significantly better than alternatives (backed by large amounts of government money). Over time, as they get used in more and more products, these LLMs can become more 'aligned' to these regimes way of thinking. There are no Chinese companies that are not part of the Chi…

Aligned how? If you download the code, it won't change under your feet.

The weights, calculated through very intensive computing, are what hold the knowledge in LLMs, the source code just executes those. These products could just update/patch their weights periodically, and no one would complain because that's not bad per se.

Re: InternLM – new open source 7B LLM

#88

Post by seanhunter »

cubefox wrote: post #81
seanhunter wrote: post #61

Earlier quoted context omitted.

All 3 text-davinci models are available on openAI's api. including 3 (which is the GPT-3.5 gen). Code-davinci-002 is a code-tuned model, You can see a nice visual summary of the relationships between the openAI models at https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr... Or the official source is https://platform.openai.com/docs/model-index-for-researchers

> All 3 text-davinci models are available on openAI's api. That's irrelevant because these are all fine-tuned. > Code-davinci-002 is a code-tuned model No, "code-tuned" isn't even a thing. It is a foundation model, which consists purely of pretreating. No fine-tuning is involved. > Or the official source is The official source says exactly what I just said.

OK perhaps I used slightly the wrong term. The docs[1] say that code-davinci-002 is "optimized for code completion tasks" though so it seems unlikely to fulfil the OPs purpose of playing around with an unaligned/sweary model which was my main point. Some of the uncensored models from huggingface would probably serve that purpose much better.

[1] see the entry for code-davinci-002 in https://platform.openai.com/docs/models/gpt-3-5

Re: InternLM – new open source 7B LLM

#89

Post by cubefox »

seanhunter wrote: post #88
cubefox wrote: post #81

Earlier quoted context omitted.

> All 3 text-davinci models are available on openAI's api. That's irrelevant because these are all fine-tuned. > Code-davinci-002 is a code-tuned model No, "code-tuned" isn't even a thing. It is a foundation model, which consists purely of pretreating. No fine-tuning is involved. > Or the official source is The official source says exactly what I just said.

OK perhaps I used slightly the wrong term. The docs[1] say that code-davinci-002 is "optimized for code completion tasks" though so it seems unlikely to fulfil the OPs purpose of playing around with an unaligned/sweary model which was my main point. Some of the uncensored models from huggingface would probably serve that purpose much better. [1] see the entry for code-davinci-002 in https://platform.openai.com/docs/m…

Code was just part of its pretraining. All other GPT-3.5 models are fine-tuned versions of code-davinci-002.

Quote:

1 code-davinci-002 is a base model, so good for pure code-completion tasks

2 text-davinci-002 is an InstructGPT model based on code-davinci-002

3 text-davinci-003 is an improvement on text-davinci-002

4 gpt-3.5-turbo-0301 is an improvement on text-davinci-003, optimized for chat

Quote end.

https://platform.openai.com/docs/model-index-for-researchers

The reason you want a base model for code completion has nothing to do with code itself, it has to do with the fact that it completes text unlike all the instruction tuned models, which expect instructions. When you have code, there aren't necessarily any instructions present. You basically want autocomplete. That's what a base model does. But that doesn't mean it doesn't work with other things apart from code. After all, all other GPT-3.5 models are just code-davinci-002 with additional instruction and RLHF fine-tuning added, and they know countless other subject areas apart from code.

I don't get why this is so hard to understand.

Re: InternLM – new open source 7B LLM

#90

Post by seanhunter »

cubefox wrote: post #89
seanhunter wrote: post #88

Earlier quoted context omitted.

OK perhaps I used slightly the wrong term. The docs[1] say that code-davinci-002 is "optimized for code completion tasks" though so it seems unlikely to fulfil the OPs purpose of playing around with an unaligned/sweary model which was my main point. Some of the uncensored models from huggingface would probably serve that purpose much better. [1] see the entry for code-davinci-002 in https://platform.openai.com/docs/m…

Code was just part of its pretraining. All other GPT-3.5 models are fine-tuned versions of code-davinci-002. Quote: 1 code-davinci-002 is a base model, so good for pure code-completion tasks 2 text-davinci-002 is an InstructGPT model based on code-davinci-002 3 text-davinci-003 is an improvement on text-davinci-002 4 gpt-3.5-turbo-0301 is an improvement on text-davinci-003, optimized for chat Quote end. https://platf…

It's not hard to understand. We just have a disagreement about something that you think is very important probably partly because you know more about this than I do. Have a nice day. Thanks for explaining.
Post reply on HN