Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How large are large language models?

gist.github.com

Post reply on HN
Print view View original

91–100 of 160 posts

Re: How large are large language models?

#91

Post by penguin_booze »

agumonkey wrote: post #55
ljoshua wrote: post #30

Less a technical comment and more just a mind-blown comment, but I still can’t get over just how much data is compressed into and available in these downloadable models. Yesterday I was on a plane with no WiFi, but had gemma3:12b downloaded through Ollama. Was playing around with it and showing my kids, and we fired history questions at it, questions about recent video games, and some animal fact questions. It wasn’t…

Intelligence is compression some say

I don't know why, but I was reminded of Douglas Hofstadter's talk: Analogy is cognition: https://www.youtube.com/watch?v=n8m7lFQ3njk&t=964s.

Re: How large are large language models?

#92

Post by thecosas »

ljoshua wrote: post #30

Less a technical comment and more just a mind-blown comment, but I still can’t get over just how much data is compressed into and available in these downloadable models. Yesterday I was on a plane with no WiFi, but had gemma3:12b downloaded through Ollama. Was playing around with it and showing my kids, and we fired history questions at it, questions about recent video games, and some animal fact questions. It wasn’t…

A neat project you (and others) might want to check out: https://kiwix.org/

Lots of various sources that you can download locally to have available offline. They're even providing some pre-loaded devices in areas where there may not be reliable or any internet access.

Re: How large are large language models?

#93

Post by [deleted] »

ljoshua wrote: post #30

Less a technical comment and more just a mind-blown comment, but I still can’t get over just how much data is compressed into and available in these downloadable models. Yesterday I was on a plane with no WiFi, but had gemma3:12b downloaded through Ollama. Was playing around with it and showing my kids, and we fired history questions at it, questions about recent video games, and some animal fact questions. It wasn’t…

[deleted]

Re: How large are large language models?

#94

Post by agumonkey »

Nevermark wrote: post #75
agumonkey wrote: post #55

Earlier quoted context omitted.

Intelligence is compression some say

Very much so! The more and faster a “mind” can infer, the less it needs to store. Think how much fewer facts a symbolic system that can perform calculus needs to store, vs. an algebraic, or just arithmetic system, to cover the same numerical problem solving space. Many orders of magnitude less. The same goes for higher orders of reasoning. General or specific subject related. And higher order reasoning vastly increas…

I don't know who else took notes by diffing their own assumptions with lectures / talks. There was a notion of what's really new compared to previous conceptual state, what adds new information.

Re: How large are large language models?

#95

Post by entropicdrifter »

MPSimmons wrote: post #44
rain1 wrote: post #36

Earlier quoted context omitted.

It's extremely interesting how powerful a language model is at compression. When you train it to be an assistant model, it's better at compressing assistant transcripts than it is general text. There is an eval which I have a lot of interested in and respect for https://huggingface.co/spaces/Jellyfish042/UncheatableEval called UncheatableEval, which tests how good of a language model an LLM is by applying it on a ran…

Agreed. It's basically lossy compression for everything it's ever read. And the quantization impacts the lossiness, but since a lot of text is super fluffy, we tend not to notice as much as we would when we, say, listen to music that has been compressed in a lossy way.

It's a bit like if you trained a virtual band to play any song ever, then told it to do its own version of the songs. Then prompted it to play whatever specific thing you wanted. It won't be the same because it kinda remembers the right thing sorta, but it's also winging it.

Re: How large are large language models?

#96

Post by exe34 »

Wowfunhappy wrote: post #79
Nevermark wrote: post #70

Earlier quoted context omitted.

It blows my mind that I can ask for 50 synonyms, instantly get a great list with great meaning summaries. Then ask for the same list sorted and get that nearly instantly, These models have a short time context for now, but they already have a huge “working memory” relative to us . It is very cool. And indicative that vastly smarter models are going to be achieved fairly easily, with new insight. Our biology has had t…

> These models [...] have a huge “working memory” relative to us. [This is] indicative that vastly smarter models are going to be achieved fairly easily, with new insight. I don't think your second sentence logically follows from the first. Relative to us, these models: - Have a much larger working memory. - Have much more limited logical reasoning skills. To some extent, these models are able to use their superior w…

> But humans are able to reason with orders of magnitude less training data.

Common belief, but false. You start learning from inside the womb. The data flow increases exponentially when you open your eyes and then again when you start manipulating things with your hands and mouth.

> When you ask a model to "think about the problem step by step" to improve its reasoning, you are basically just giving it more opportunities to draw on its huge memory bank and try to put things together.

We do the same with children. At least I did it to my classmates when they asked me for help. I'd give them a hint, and ask them to work it out step by step from there. It helped.

Re: How large are large language models?

#97

Post by exe34 »

antonvs wrote: post #88
Wowfunhappy wrote: post #79

Earlier quoted context omitted.

> These models [...] have a huge “working memory” relative to us. [This is] indicative that vastly smarter models are going to be achieved fairly easily, with new insight. I don't think your second sentence logically follows from the first. Relative to us, these models: - Have a much larger working memory. - Have much more limited logical reasoning skills. To some extent, these models are able to use their superior w…

> Have much more limited logical reasoning skills. Relative to the best humans, perhaps, but I seriously doubt this is true in general. Most people I work with couldn’t reason nearly as well through the questions I use LLMs to answer. It’s also worth keeping in mind that having a different approach to reasoning is not necessarily equivalent to a worse approach. Watch out for cherry-picking the cons of its approach an…

> Relative to the best humans,

For some reason, the bar for AI is always against the best possible human, right now.

Re: How large are large language models?

#98

Post by llSourcell »

netcan wrote: post #32
smokel wrote: post #16

Earlier quoted context omitted.

Simply add images and video, and these estimates start to sound like the "640 KB should be enough for everyone". After that, make the robots explore and interact with the world by themselves, to fetch even more data. In all seriousness, adding image and interaction data will probably be enormously useful, even for generating text.

Like both will be done. Idk what the roi is on adding video data to the text models, but it's presumably lower than text. There are just a lot of avenues to try at this point.

no its not lower than text, its higher ROI than text for understanding the physics of the world, which is exactly what videos are better at than text when it comes to training data

Re: How large are large language models?

#99

Post by swyx »

ljoshua wrote: post #30

Less a technical comment and more just a mind-blown comment, but I still can’t get over just how much data is compressed into and available in these downloadable models. Yesterday I was on a plane with no WiFi, but had gemma3:12b downloaded through Ollama. Was playing around with it and showing my kids, and we fired history questions at it, questions about recent video games, and some animal fact questions. It wasn’t…

the study of language models from an information theory/compression POV is a small field but increasingly impt for efficiency/scaling - we did a discussion about this today https://www.youtube.com/watch?v=SWIKyLSUBIc&t=2269s

Re: How large are large language models?

#100

Post by swyx »

angusturner wrote: post #56
Workaccount2 wrote: post #49

Earlier quoted context omitted.

I don't like the term "compression" used with transformers because it gives the wrong idea about how they function. Like that they are a search tool glued onto a .zip file, your prompts are just fancy search queries, and hallucinations are just bugs in the recall algo. Although strictly speaking they have lots of information in a small package, they are F-tier compression algorithms because the loss is bad, unpredict…

There is an excellent talk by Jack Rae called “compression for AGI”, where he shows (what I believe to be) a little known connection between transformers and compression; In one view, you can view LLMs as SOTA lossless compression algorithms, where the number of weights don’t count towards the description length. Sounds crazy but it’s true.

his talk here https://www.youtube.com/watch?v=dO4TPJkeaaU

and his last before departing for Meta Superintelligence https://www.youtube.com/live/U-fMsbY-kHY?si=_giVEZEF2NH3lgxI...

Post reply on HN