Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big LLMs weights are a piece of history

antirez.com

Post reply on HN
Print view View original

111–120 of 222 posts

Re: Big LLMs weights are a piece of history

#111

Post by dmos62 »

lblume wrote: post #68
dmos62 wrote: post #50

Enjoy the insight, but the title makes my eye twitch. How about "LLM weights are pieces of history"?

Small LLM weights are not really interesting though. I am currently training GPT-2 small sized models for a scientific project right, and their world models are just not good enough to generate any kind of real insight about the world it was trained in except for corpus biases.

A collection of newspapers is generally a better source than a single leaflet, but even a leaflet is a piece of history.

Re: Big LLMs weights are a piece of history

#112

Post by badlibrarian »

ben_w wrote: post #78
badlibrarian wrote: post #31

Earlier quoted context omitted.

I've sat in more than one board meeting watching them take 20 minutes to land on t-shirt sizes. The greatest enterprise sales minds of our generation...

I've seen things you people wouldn't believe. I’ve seen corporate slogans fired off from the shoulders of viral creatives. Synergy-beams glittering in the darkness of org charts. Thought leadership gone rogue… All these moments will be lost to NDAs and non-disparagement clauses, like engagement metrics in a sea of pivot decks. Time to leverage.

... destroyed by madness, starving hysterical! Buying weed in a store then meeting with someone off Craiglist to score eggs.

Re: Big LLMs weights are a piece of history

#113

Post by cowsaymoo »

heyjamesknight wrote: post #99
t_mann wrote: post #23

Earlier quoted context omitted.

Big LLM is too long as a name. We should agree on calling them BLLMs. Surely everyone is going to remember what the letters stand for.

I want to apologize for this joke in advance. It had to be done. We could take a page from Trump’s book and call them “Beautiful” LLMs. Then we’d have “Big Beautiful LLMs” or just “BBLs” for short. Surely that wouldn’t cause any confusion when Googling.

Weirdly enough, the ITU already chose the superlative for the bigliest radio frequency band to be Tremendous:

- Extremely Low Frequency (ELF)

- Super Low Frequency (SLF)

- Ultra Low Frequency (ULF)

- Very Low Frequency (VLF)

- Low Frequency (LF)

- Medium Frequency (MF)

- High Frequency (HF)

- Very High Frequency (VHF)

- Ultra High Frequency (UHF)

- Super High Frequency (SHF)

- Extremely High Frequency (EHF)

- Tremendously High Frequency (THF)

Maybe one day some very smart people will make Tremendously Large Language Models. They will be very large and need a lot of computer. And then you'll have the Extremely Small Language Model. They are like nothing.

https://en.wikipedia.org/wiki/Radio_frequency?#Frequency_ban...

Re: Big LLMs weights are a piece of history

#114

Post by thih9 »

TZubiri wrote: post #87
intellectronica wrote: post #11

I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.

Doesn't the first L in LLM mean large already? It's like saying Automated ATM. Whoever wrote it barely knows what the acronym means. This whole article feels like written by someone who doesn't understand the subject matter at all

We’re fine with “The big friendly giant” and the sahara desert (“desert desert”); big llm could join the family of pleonasms.

https://en.m.wikipedia.org/wiki/Pleonasm

Re: Big LLMs weights are a piece of history

#115

Post by antirez »

dstroot wrote: post #107

Isn’t big LLM training data actually the most analogous to the internet archive? Shouldn’t the title be “Big LLM training data is a piece of history”? Especially at this point in history since a large portion of internet data going forward will be LLM generated and not human generated? It’s kind of the last snapshot of human-created content.

The problem is, where is this 20T tokens that are being used for this task? No way to access them. I hope that at least OpenAI and a few more have solid historical storage of the tokens they collect.

Re: Big LLMs weights are a piece of history

#118

Post by swyx »

latexr wrote: post #77
intellectronica wrote: post #11

I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.

Name them like clothing sizes: XXLLM, XLLM, LLM, MLM, SLM, XSLM XXSLM.

i did this!

XXLLM: ~1T (GPT4/4.5, Claude Opus, Gemini Pro)

XLLM: 300~500B (4o, o1, Sonnet)

LLM: 20~200B (4o, GPT3, Claude, Llama 3 70B, Gemma 27B)

~~zone of emergence~~

MLM: 7~14B (4o-mini, Claude Haiku, T5, LLaMA, MPT)

SLM: 1~3B (GPT2, Replit, Phi, Dall-E)

~~zone of generality~~

XSLM: 4XSLM: https://x.com/swyx/status/1679241722709311490

Re: Big LLMs weights are a piece of history

#119

Post by samstave »

radlad wrote: post #49
samstave wrote: post #26

Earlier quoted context omitted.

I want a tiny_phone_based LLM to do thought tracking and comms awareness.. I actually applied to YC in like ~2014 or such for thus; -JotPlot - I wanted a timeline for basically giving a histo timeline of comms btwn me and others - such that I had a sankey-ish diagram for when and whom and via method I spoke with folks and then each node eas the message, call, text, meta links... I think its still viable - but my thou…

I had a thought that I think some people value social media (e.g. Facebook) essentially for this. Like giving up your Facebook profile means giving up your history or family tree or even your memories. So in that sense, maybe people would prefer a private alternative.

I read this in Sam Wattersons voice with a pipe abt maybey an inch from his beard,

(Fyi I was a designer at fb and while it was luxious I still hated what I saw in zucks eyes every morn when I passed him.

Super diff from Andy Grove at intel where for whateveer reason we were in the sam oee schekdule

(That was me typing with eues ckised as a test (to myself, typos abound

Re: Big LLMs weights are a piece of history

#120

Post by miki123211 »

xanderlewis wrote: post #91
intellectronica wrote: post #11

I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.

And the US ‘small’ LLMs will actually be slightly larger than the ‘large’ LLMs in the UK.

> The UK

You mean the EU, right? The UK isn't covered by the AI act.

/s

Post reply on HN