Live data from Hacker News

Open source LLM with 32k Context Length

blog.abacus.ai

11–20 of 31 posts

Re: Open source LLM with 32k Context Length

#11

It seems this is built on LLAMA. Did meta change the license to make it open source now? It still seems to be showing otherwise in the repo. Edit: No mention of it being open source in the linked article. Maybe the title here is just wrong? @dang

Llama 2 is open source-ish. Weights are freely available and can be commercially used, but only if you have less than 700m users and agree to some "don't do naughty things" terms.

[deleted]

Re: Open source LLM with 32k Context Length

#12

It seems this is built on LLAMA. Did meta change the license to make it open source now? It still seems to be showing otherwise in the repo. Edit: No mention of it being open source in the linked article. Maybe the title here is just wrong? @dang

Llama 2 is open source-ish. Weights are freely available and can be commercially used, but only if you have less than 700m users and agree to some "don't do naughty things" terms.

"open source-ish" sounds like the perfect way to massively profitable future litigations.

Also "don't do naughty things", is there a chart for that? How is that defined, is it part of the non-existing license?

Re: Open source LLM with 32k Context Length

#13
It's probably too new for anyone to have integrated this into text-generation-webui / Gradio? I've been looking for a large context LLM (self-hosted or not) for a project, and as a European I unfortunately don't have access to Anthropic's Claude API yet.

Re: Open source LLM with 32k Context Length

#14
post #13

It's probably too new for anyone to have integrated this into text-generation-webui / Gradio? I've been looking for a large context LLM (self-hosted or not) for a project, and as a European I unfortunately don't have access to Anthropic's Claude API yet.

It's just Llama 2 w/ rotary encoding fine tuned to 32k. It should work fine.

Re: Open source LLM with 32k Context Length

#15

It seems this is built on LLAMA. Did meta change the license to make it open source now? It still seems to be showing otherwise in the repo. Edit: No mention of it being open source in the linked article. Maybe the title here is just wrong? @dang

From memory llama 2 license does allow tuned models with suitable credit & license inclusion. The restricted using it to train other models though (a bit like people use gpt4 to generate question/answer pairs to train their models)

Re: Open source LLM with 32k Context Length

#17

It seems this is built on LLAMA. Did meta change the license to make it open source now? It still seems to be showing otherwise in the repo. Edit: No mention of it being open source in the linked article. Maybe the title here is just wrong? @dang

Llama 2 is open source-ish. Weights are freely available and can be commercially used, but only if you have less than 700m users and agree to some "don't do naughty things" terms.

Nope. It's limited by 700m monthly-active users at the time Llama2 was released, a weird catch clause for a handful of Meta competitor companies. The license doesn't satisfy OSS requirements, but it is quite reasonable.

https://ai.meta.com/llama/license/ https://ai.meta.com/llama/use-policy/

Re: Open source LLM with 32k Context Length

#18

Earlier quoted context omitted.

Llama 2 is open source-ish. Weights are freely available and can be commercially used, but only if you have less than 700m users and agree to some "don't do naughty things" terms.

There is no open source-ish. It either protects the fundamental freedoms or it...doesn't.

Sure there is, BSD is bad because xyz, GPL is bad because zyx. That said Llama restrictions are rather harsh and you are not allowed to improve other models with it. So no freedom there just some ok beer.

Re: Open source LLM with 32k Context Length

#19

Earlier quoted context omitted.

Llama 2 is open source-ish. Weights are freely available and can be commercially used, but only if you have less than 700m users and agree to some "don't do naughty things" terms.

There is no open source-ish. It either protects the fundamental freedoms or it...doesn't.

Open source is both a colloquial term for available/modifiable/distributable code (like Llama2), and a strict OSI-approved list of licenses. I'd say opensource-ish is a great fit here.

Edit: this is in fact fairly interesting discussion because LLM is a new breed of digital products. Meta's terms are practical for limiting the usage for commercial applications, and they are designed to protect the general population. It's not the worn out "protecting us from ourselves", its actually preventing Llama users from harming non-users. Yes, we can be jaded and say it's about protecting the brand and dissociating from bad actors. My point is that it's hard to apply usual arguments for open source and freedom of computing, when you're defending rights of people who want to harm other people.

Re: Open source LLM with 32k Context Length

#20
post #16

Does anyone know if larger context lengths are inherent worse at other task? i.e. all other things being equal is a 8k model better at math than a 32k model

There’s a couple models on huggingface that uses NTK/linear RoPE that you can play with. Vicuna and WizardLM both have a 16K context model. The biggest issue is that if you go to really high context, it sometimes does these weird repetitions. But to be fair, I only have tried the quantized models and 13B (highest I can run locally). Not sure if the repetition are an artifact of the rope or quantization or both.
Post reply on HN