Imagine future historians piecing together our culture from hallucinated AI memories - inaccurate, sure, but maybe even more fascinating than reality itself.
Big LLMs weights are a piece of history
181–190 of 222 posts
Re: Big LLMs weights are a piece of history
#182Earlier quoted context omitted.
And the US ‘small’ LLMs will actually be slightly larger than the ‘large’ LLMs in the UK.
I wonder how does the skinnies get dressed oversea: I wear European S which translate to XXS in the US, but there’s many people skinnier than me, still within a “normal" BMI. Do they have to find XXXS? Do they wear oversized clothes? Choosing trousers is way easier because the system of cm/inches of length+perimeter correspond to real values.
> Choosing trousers is way easier because the system of cm/inches of length+perimeter correspond to real values.
They're not merely real values, they're also rational.
Re: Big LLMs weights are a piece of history
#183Earlier quoted context omitted.
It's funny you say that, but when travelling abroad I wondered how Europeans and Japanese stay sufficiently hydrated.
For healthy adults, thirst is a perfectly adequate guide to hydration needs. Historically normal patterns of drinking - e.g. water with meals and a few cups of tea or coffee in between - are perfectly sufficient unless you're doing hard physical labour or spending long periods of time outdoors in hot weather. The modern American preoccupation with constantly drinking water is a peculiar cultural phenomenon with no sc…
Try getting a kidney stone and then find out if adequate hydration is what you want to squeak by with.
Re: Big LLMs weights are a piece of history
#184I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.
I'd prefer to see olive sizes get a renaissance. I was always amused by Super Colossal when following my mom around a store as a little kid. From a random web search, it seems the sizes above Large are: Extra Large, Jumbo, Extra Jumbo, Giant, Colossal, Super Colossal, Mammoth, Super Mammoth, Atlas.
How about wine bottle sizes since we're "bottling" a "distillation" of information...
Re: Big LLMs weights are a piece of history
#185Earlier quoted context omitted.
I'd prefer to see olive sizes get a renaissance. I was always amused by Super Colossal when following my mom around a store as a little kid. From a random web search, it seems the sizes above Large are: Extra Large, Jumbo, Extra Jumbo, Giant, Colossal, Super Colossal, Mammoth, Super Mammoth, Atlas.
How about wine bottle sizes since we're "bottling" a "distillation" of information... https://en.wikipedia.org/wiki/Wine_bottle#Sizes
To get pedantic, wine is not a product of distillation.
Re: Big LLMs weights are a piece of history
#186Earlier quoted context omitted.
I want to apologize for this joke in advance. It had to be done. We could take a page from Trump’s book and call them “Beautiful” LLMs. Then we’d have “Big Beautiful LLMs” or just “BBLs” for short. Surely that wouldn’t cause any confusion when Googling.
Weirdly enough, the ITU already chose the superlative for the bigliest radio frequency band to be Tremendous: - Extremely Low Frequency (ELF) - Super Low Frequency (SLF) - Ultra Low Frequency (ULF) - Very Low Frequency (VLF) - Low Frequency (LF) - Medium Frequency (MF) - High Frequency (HF) - Very High Frequency (VHF) - Ultra High Frequency (UHF) - Super High Frequency (SHF) - Extremely High Frequency (EHF) - Tremend…
"The Overwhelmingly Large Telescope (OWL) was a conceptual design by the European Southern Observatory (ESO) organisation for an extremely large telescope, which was intended to have a single aperture of 100 metres in diameter. Because of the complexity and cost of building a telescope of this unprecedented size, ESO has decided to focus on the 39-metre diameter Extremely Large Telescope instead."
https://en.m.wikipedia.org/wiki/Overwhelmingly_Large_Telesco...
Re: Big LLMs weights are a piece of history
#187“We should regard the Internet Archive as one of the most valuable pieces of modern history; instead, many companies and entities make the chances of the Archive to survive, and accumulate what otherwise will be lost, harder and harder. I understand that the Archive headquarters are located in what used to be a church: well, there is no better way to think of it than as a sacred place.” Amen. There is an active effor…
Yup! We're here and looking to do good work with Cultural Heritage and Research Organizations in Europe. I'm very happy to be working with the Internet Archive once again after a 20 year long break. https://www.stichtinginternetarchive.nl/
Congratulations!
Re: Big LLMs weights are a piece of history
#188Earlier quoted context omitted.
How about wine bottle sizes since we're "bottling" a "distillation" of information... https://en.wikipedia.org/wiki/Wine_bottle#Sizes
To get pedantic, wine is not a product of distillation.
That almost makes the metaphor more apt. Wine is the real deal, and brandy is the distilled approximation.
Re: Big LLMs weights are a piece of history
#189Enjoy the insight, but the title makes my eye twitch. How about "LLM weights are pieces of history"?
Small LLM weights are not really interesting though. I am currently training GPT-2 small sized models for a scientific project right, and their world models are just not good enough to generate any kind of real insight about the world it was trained in except for corpus biases.
Small large language models? This sounds like the apocryphal headline when a spiritualist with dwarfism escaped prison: "Small medium at large." Do you also have some dehydrated water and a secure key escrow system?
Re: Big LLMs weights are a piece of history
#190Earlier quoted context omitted.
Apple Intelligence has an LLM that runs locally on the iPhone (15 Pro and up). But the quality of Apple Intelligence shows us what happens when you use a tiny ultra-low-wattage LLM. There’s a whole subreddit dedicated to its notable fails: https://www.reddit.com/r/AppleIntelligenceFail/top/?t=all One example of this is “Sorry I was very drunk and went home and crashed straight into bed” being summarized by Apple Inte…
I think the real problem with LLMs is we have deterministic expectations of non-deterministic tools. We’ve been trained to expect that the computer is correct. Personally, I think the summaries of alerts is incredibly useful. But my expectation of accuracy for a 20 word summary of multiple 20-30 word summaries is tempered by the reality that’s there’s gonna be issues given the lack of context. The point of the summar…
Determinism isn't the issue though. Many responses are fine. The displayed one is bad, whether chosen deterministically or not. Some alternatives:
- Passed out drunk
- Crashed in bed
- Slacking because drunk
...
The issue isn't a lack of context; it's that even the available context was handled poorly.