How large are large language models?
gist.github.com
How large are large language models?
1–10 of 160 posts
Re: How large are large language models?
#2Meta: The inclusion of the current year ("(2025)") in the title is strange, even though it's in the actual title of the linked-to post, repeating it here makes me look around for the time machine controls.
Re: How large are large language models?
#3How big are those in terms of size on disk and VRAM size?
Something like 1.61B just doesn't mean much to me since I don't know much about the guts of LLMs. But I'm curious about how that translates to computer hardware -- what specs would I need to run these? What could I run now, what would require spending some money, and what I might hope to be able to run in a decade?
Re: How large are large language models?
#4>None of this document was not written by AI
I think in these scenarios, articles should include the prompt and generating model.
Re: How large are large language models?
#5Deepseek v1 is ~670Bn which is ~1.4TB physical.
All digitized books ever written/encoded compress to a few TB. The public web is ~50TB. I think a usable zip of all english electronic text publicly available would be on O(100TB). So we're at about 1% of that in model size, and we're in a diminishing-returns area of training -- ie., going to >1% has not yielded improvements (cf. gpt4.5 vs 4o).
This is why compute spend is moving to inference time with "reasoning" models. It's likely we're close to diminshing returns on inference-time compute now too, hence agents whereby (mostly,) deterministic tools are supplementing information /capability into the system.
I think to get any more value out of this model class, we'll be looking at domain-specific specialisation beyond instruction fine-tuning.
I'd guess targeting 1TB inference-time VRAM would be a reasonable medium-term target for high quality open source models -- that's within the reach of most SMEs today. That's about 250bn params.
Re: How large are large language models?
#6>None of this document was not written by AI I think in these scenarios, articles should include the prompt and generating model.
You are absolutely right! The AI slop is getting out of control.
Re: How large are large language models?
#7How big are those in terms of size on disk and VRAM size? Something like 1.61B just doesn't mean much to me since I don't know much about the guts of LLMs. But I'm curious about how that translates to computer hardware -- what specs would I need to run these? What could I run now, what would require spending some money, and what I might hope to be able to run in a decade?
At 1byte/param that's 1.6GB (f8), at 2 bytes (f16) that's 2.3GB -- but there's other space costs beyond loading the parameters for the GPU. So a rule of thumb is ~4x parameter count. So round up, 2B -> 2*4 = 8GB VRAM
Re: How large are large language models?
#8Deepseek v1 is ~670Bn which is ~1.4TB physical. All digitized books ever written/encoded compress to a few TB. The public web is ~50TB. I think a usable zip of all english electronic text publicly available would be on O(100TB). So we're at about 1% of that in model size, and we're in a diminishing-returns area of training -- ie., going to >1% has not yielded improvements (cf. gpt4.5 vs 4o). This is why compute spend…
> All digitized books ever written/encoded compress to a few TB. The public web is ~50TB. I think a usable zip of all english electronic text publicly available would be on O(100TB).
Where you getting these numbers from? Interested to see how that's calculated.
I read somewhere, but cannot find the source anymore, that all written text prior to this century was approx 50MB. (Might be misquoted as don't have source anymore).
Re: How large are large language models?
#9Deepseek v1 is ~670Bn which is ~1.4TB physical. All digitized books ever written/encoded compress to a few TB. The public web is ~50TB. I think a usable zip of all english electronic text publicly available would be on O(100TB). So we're at about 1% of that in model size, and we're in a diminishing-returns area of training -- ie., going to >1% has not yielded improvements (cf. gpt4.5 vs 4o). This is why compute spend…
> All digitized books ever written/encoded compress to a few TB. The public web is ~50TB. I think a usable zip of all english electronic text publicly available would be on O(100TB). Where you getting these numbers from? Interested to see how that's calculated. I read somewhere, but cannot find the source anymore, that all written text prior to this century was approx 50MB. (Might be misquoted as don't have source an…
Maybe prior to the prior century, and even then I smell a lot of bullshit. I mean, just look at the Project Gutenberg. Even plaintext only, even compressed.
Re: How large are large language models?
#10>None of this document was not written by AI I think in these scenarios, articles should include the prompt and generating model.
I don’t think the author knows that double negatives in English in a sentence like this cancel, not reinforce, each other.