Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big LLMs weights are a piece of history

antirez.com

Post reply on HN
Print view View original

161–170 of 222 posts

Re: Big LLMs weights are a piece of history

#161

Post by stogot »

blmurch wrote: post #150
dr_dshiv wrote: post #64

“We should regard the Internet Archive as one of the most valuable pieces of modern history; instead, many companies and entities make the chances of the Archive to survive, and accumulate what otherwise will be lost, harder and harder. I understand that the Archive headquarters are located in what used to be a church: well, there is no better way to think of it than as a sacred place.” Amen. There is an active effor…

Yup! We're here and looking to do good work with Cultural Heritage and Research Organizations in Europe. I'm very happy to be working with the Internet Archive once again after a 20 year long break. https://www.stichtinginternetarchive.nl/

What kind of volunteer help can the community do?

Re: Big LLMs weights are a piece of history

#162

Post by ac29 »

floriannn wrote: post #151
jgalt212 wrote: post #105

Earlier quoted context omitted.

It's funny you say that, but when travelling abroad I wondered how Europeans and Japanese stay sufficiently hydrated.

Is this a thing about how restaurants in some European countries charge for water?

Its a joke about Americans carrying around giant water bottles

Re: Big LLMs weights are a piece of history

#163

Post by jfengel »

ben_w wrote: post #82
jfengel wrote: post #57

Earlier quoted context omitted.

LLM 95, LLM 98, LLM Millennium Edition, LLM NT, LLM XP, LLM 2000, LLM 7 I really appreciated the way they managed to come up with a new naming scheme each time, usually used exactly once.

Could always go with the Bungie approach for the Marathon series: LLM, LLM2, LLM∞, ℵ₁ — https://alephone.lhowon.org (Obviously ∞ is for the actual singularity, and ℵ₁ is the thing after that).

Are you sure that ℵ1 is the thing after that?

https://en.m.wikipedia.org/wiki/Continuum_hypothesis

;-)

Re: Big LLMs weights are a piece of history

#164

Post by teleforce »

I really like the narative that now LLM is the conserving human knowledge that otherwise would be lost forever in the form of its weights in a kind of a lossy compression.

Personally I'd like that if all the knowledge and information (K & I) are readily available and accessible (pretty sure most of the prople share the same sentiment), despite the consistent business decisions from the copyright holders to hoard their K & I by putting everything behind paywalls and/or registration (I'm looking at you Apple and X/Twitter). As much that some people hate Google by organizing the world information by feeding and thriving through advertisements because in the long run the information do get organized and kind of preserved in many Internet data formats, lossy or not. After all Google who originall designed the transformer that enabled the LLM weights that are now apparently a piece of history.

Re: Big LLMs weights are a piece of history

#165

Post by jart »

xpe wrote: post #139
jart wrote: post #98

Earlier quoted context omitted.

We never would have been able to create science, if it weren't for focusing on the kinds of thinking that can be made logical. There's a big difference. What you're doing, with this whole "let's make a bullshit word logical" is more similar to medieval scholasticism, which was a vain attempt at verbal precision. https://justine.lol/dox/english.txt

Yikes, maybe we can take a step back? I'm not sure where this is coming from, frankly. One anodyne summary of my comment above would be: > Let's think and communicate more clearly regarding intelligence. Stuart Russell offers a nice definition: an agent's ability to do a defined task. Maybe something about my comment got you riled up? What was it? You wrote: > What you're doing, with this whole "let's make a bullshit…

I just wanted to share an essay I liked. I didn't think you'd pay it much mind. But I can see now that you are a person devoted to science. If you want to know what I believe, I think computers in the 50's were intelligent. I think gpt2 probably qualified as agi if you take the meaning of the acronym literally. At this point we've blown so far past all expectations in terms of intelligence that I've come to agree with Karpathy that the time has come to start moving the goalposts to other words, like agency, since agents are an unsolved problem, and agency is proving to possibly be more important/powerful/rare/difficult than intelligence.

I reacted negatively to the idea earlier that agency should be considered an aspect of intelligence. I think separating the concepts helps me better understand people, their unique strengths, and puzzles like why sometimes people who aren't geniuses who know everything and can rotate complex shapes are sometimes very successful, but most importantly, why LLMs continue to feel like they're lacking something, compared to people, even though they're so outrageously intelligent. It's one thing to be smart, another thing entirely to be useful.

Re: Big LLMs weights are a piece of history

#166

Post by intrasight »

tonyhart7 wrote: post #21
intellectronica wrote: post #11

I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.

can we have tiny LLM that can run on smartphone now

I expect that the phone will only do the prompt parsing

Re: Big LLMs weights are a piece of history

#167

Post by throwaway314155 »

andix wrote: post #129
visarga wrote: post #53

Earlier quoted context omitted.

LLMs are much easier to port than software. They are just a big blob of numbers and a few math operations.

I think software is rather easy to archive. Emulators are they key. Nearly every platform from the past can be emulated on a modern arm/x86 Linux/windows system. Arm/x86/linux/windows are ubiquitous, even if they might fade away there will be emulators around for a long time. With future compute power it should be no problem to just use nested emulation, to run old emulators on an emulated x86/linux.

> I think software is rather easy to archive.

* assuming someone else already spent tremendous effort to develop an emulator for your binary's target that is 100% accurate...

Re: Big LLMs weights are a piece of history

#168

Post by benatkin »

cowsaymoo wrote: post #113
heyjamesknight wrote: post #99

Earlier quoted context omitted.

I want to apologize for this joke in advance. It had to be done. We could take a page from Trump’s book and call them “Beautiful” LLMs. Then we’d have “Big Beautiful LLMs” or just “BBLs” for short. Surely that wouldn’t cause any confusion when Googling.

Weirdly enough, the ITU already chose the superlative for the bigliest radio frequency band to be Tremendous: - Extremely Low Frequency (ELF) - Super Low Frequency (SLF) - Ultra Low Frequency (ULF) - Very Low Frequency (VLF) - Low Frequency (LF) - Medium Frequency (MF) - High Frequency (HF) - Very High Frequency (VHF) - Ultra High Frequency (UHF) - Super High Frequency (SHF) - Extremely High Frequency (EHF) - Tremend…

TLLM is close to TLM

Re: Big LLMs weights are a piece of history

#169

Post by jart »

_ea1k wrote: post #143
visarga wrote: post #53

Earlier quoted context omitted.

LLMs are much easier to port than software. They are just a big blob of numbers and a few math operations.

Indeed. In 50 years, loading the weights and doing math should be much easier than getting some 50 year old piece of cuda code to work. Then again, CPUs will be fast enough that you'd probably just emulate amd64 and run it as CPU-only.

llamafiles run natively on both amd64 and arm64. It's difficult to imagine both of them not being in play fifty years hence. There's definitely no hope for the cuda module in the future. We have enough difficulties getting it to work today. That's why cpu mode is the default.

Re: Big LLMs weights are a piece of history

#170

Post by taneq »

saltcured wrote: post #103
intellectronica wrote: post #11

I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.

I'd prefer to see olive sizes get a renaissance. I was always amused by Super Colossal when following my mom around a store as a little kid. From a random web search, it seems the sizes above Large are: Extra Large, Jumbo, Extra Jumbo, Giant, Colossal, Super Colossal, Mammoth, Super Mammoth, Atlas.

Needs more superlatives. “Biggest” < “Extra Biggest” < “Maximum Biggest”. :D
Post reply on HN