Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Big LLMs weights are a piece of history

antirez.com

Post reply on HN
Print view View original

131–140 of 222 posts

Re: Big LLMs weights are a piece of history

#131

Post by jdietrich »

jgalt212 wrote: post #105
xanderlewis wrote: post #91

Earlier quoted context omitted.

And the US ‘small’ LLMs will actually be slightly larger than the ‘large’ LLMs in the UK.

It's funny you say that, but when travelling abroad I wondered how Europeans and Japanese stay sufficiently hydrated.

For healthy adults, thirst is a perfectly adequate guide to hydration needs. Historically normal patterns of drinking - e.g. water with meals and a few cups of tea or coffee in between - are perfectly sufficient unless you're doing hard physical labour or spending long periods of time outdoors in hot weather. The modern American preoccupation with constantly drinking water is a peculiar cultural phenomenon with no scientific basis.

Re: Big LLMs weights are a piece of history

#132

Post by varispeed »

intellectronica wrote: post #11

I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.

Then there will be "decaf LLM"

Re: Big LLMs weights are a piece of history

#133

Post by dr_dshiv »

badlibrarian wrote: post #76
dr_dshiv wrote: post #64

“We should regard the Internet Archive as one of the most valuable pieces of modern history; instead, many companies and entities make the chances of the Archive to survive, and accumulate what otherwise will be lost, harder and harder. I understand that the Archive headquarters are located in what used to be a church: well, there is no better way to think of it than as a sacred place.” Amen. There is an active effor…

Anyone who takes even an hour to audit anything about the Internet Archive will soon come to a very sad conclusion. The physical assets are stored in the blast radius of an oil refinery. They don't have air conditioning. Take the tour and they tell you the site runs slower on hot days. Great mission, but atrociously managed. Under attack for a number of reasons, mostly absurd. But a few are painfully valid.

Their yearly budget is less than the budget of just the SF library system.

Re: Big LLMs weights are a piece of history

#134

Post by TZubiri »

thih9 wrote: post #114
TZubiri wrote: post #87

Earlier quoted context omitted.

Doesn't the first L in LLM mean large already? It's like saying Automated ATM. Whoever wrote it barely knows what the acronym means. This whole article feels like written by someone who doesn't understand the subject matter at all

We’re fine with “The big friendly giant” and the sahara desert (“desert desert”); big llm could join the family of pleonasms. https://en.m.wikipedia.org/wiki/Pleonasm

When it's a different language it's fine.

Re: Big LLMs weights are a piece of history

#135

Post by Guthur »

kelseyfrog wrote: post #29
Guthur wrote: post #28

Earlier quoted context omitted.

And so what would the point be of anyone actually posting on the internet if no one actually visits the sites because large corps have essentially stolen and monetized the whole thing. And I'm sure they have or will have the ability to influence the responses so you only see what they want you to see.

That's the next step after algorithmic content feeds - algorithmic/generated comment sections. Imagine seeing an entirely different conversation happening just to get you to buy a product. A product like Coca-Cola. Imagine scrolling through a comment section that feels tailor-made to your tastes, seamlessly guiding you to an ice-cold Coca-Cola. You see people reminiscing about their best summer memories—each one feat…

Or, war is good peace is bad, nuclear war is winnable, don't worry and start loving the bomb. The enemy are not human anyway, your life will be better with fewer people around.

Look at the people who want to control this, they do not want to sell you Coke.

Re: Big LLMs weights are a piece of history

#136

Post by badlibrarian »

dr_dshiv wrote: post #133
badlibrarian wrote: post #76

Earlier quoted context omitted.

Anyone who takes even an hour to audit anything about the Internet Archive will soon come to a very sad conclusion. The physical assets are stored in the blast radius of an oil refinery. They don't have air conditioning. Take the tour and they tell you the site runs slower on hot days. Great mission, but atrociously managed. Under attack for a number of reasons, mostly absurd. But a few are painfully valid.

Their yearly budget is less than the budget of just the SF library system.

Then maybe they should've figured out how to keep hard drives in a climate controlled environment before they decided to launch a bank.

https://ncua.gov/newsroom/press-release/2016/internet-archiv...

Re: Big LLMs weights are a piece of history

#138

Post by Kiro »

TZubiri wrote: post #87
intellectronica wrote: post #11

I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.

Doesn't the first L in LLM mean large already? It's like saying Automated ATM. Whoever wrote it barely knows what the acronym means. This whole article feels like written by someone who doesn't understand the subject matter at all

Yes, that's the point of the comment and the whole discussion here. LLMs are already Large so what should the prefix be? Big LLM is a strong contender. I'm also pretty sure the creator of redis is not "someone who doesn't understand the subject matter at all".

Re: Big LLMs weights are a piece of history

#139

Post by xpe »

jart wrote: post #98
xpe wrote: post #30

Earlier quoted context omitted.

I regularly pushback against casual uses of the word “intelligence”. First, there is no objective dividing line. It is a matter of degree relative to something else. Any language that suggests otherwise should be refined or ejected from our culture and language. Language’s evolution doesn’t have to be a nosedive. Second, there are many definitions of intelligence; some are more useful than others. Along with many, I…

We never would have been able to create science, if it weren't for focusing on the kinds of thinking that can be made logical. There's a big difference. What you're doing, with this whole "let's make a bullshit word logical" is more similar to medieval scholasticism, which was a vain attempt at verbal precision. https://justine.lol/dox/english.txt

Yikes, maybe we can take a step back? I'm not sure where this is coming from, frankly. One anodyne summary of my comment above would be:

> Let's think and communicate more clearly regarding intelligence. Stuart Russell offers a nice definition: an agent's ability to do a defined task.

Maybe something about my comment got you riled up? What was it?

You wrote:

> What you're doing, with this whole "let's make a bullshit word logical" is more similar to medieval scholasticism, which was a vain attempt at verbal precision.

Again, I'm not quite sure what to say. You suggest my comment is like a medieval scholar trying to reconcile dogma with philosophy? Wow. That's an uncharitable reading of my comment.

I have five points in response. First, the word intelligence need not be a "bullshit word", though I'm not sure what you mean by the term. One of my favorite definitions of bullshitting comes from "On Bullshit" by Harry Frankfurt:

> Frankfurt determines that bullshit is speech intended to persuade without regard for truth. The liar cares about the truth and attempts to hide it; the bullshitter doesn't care whether what they say is true or false. - Wikipedia

Second, I'm trying to clarify the term intelligence by breaking it into parts. I wouldn't say I'm trying to make it "logical" (in the sense of being about logic or deduction). Maybe you mean "formal"?

Third, regarding the "what you're doing" part... this isn't just me. Many people both clarify the concept of intelligence and explain why doing so is important.

Fourth, are you saying it is impossible to clarify the meaning of intelligence? Why? Not worth the trouble?

Fifth, have you thought about a definition of intelligence that you think is sensible? Does your definition steer people away from confusion?

You also wrote:

> We never would have been able to create science, if it weren't for focusing on the kinds of thinking that can be made logical.

I think you mean _testable_, not _logical_. Yes, we agree, scientists should run experiments on things that can be tested.

Russell's definition of intelligence is testable by defining a task and a quality metric. This is already a big step up from an unexamined view of intelligence, which often has some arbitrary threshold.* It allows us to see a continuum from, say, how a bacteria finds food, to how ants collaborate, to how people both build and use tools to solve problems. It also teases out sentience and moral worth so we're not mixing them up with intelligence. These are simple, doable, and worthwhile clarifications.

Finally, I read your quote from Dijkstra. In my reading, Dijkstra's main point is that natural language is a poor programming interface due to its ambiguity. Ok, fair. But what is the connection to this thread? Does it undercut any of my arguments? How?

* A common problem when discussing intelligence involves moving the goal post. Whatever quality bar is implied has a tendency to creep upwards over time.*

Re: Big LLMs weights are a piece of history

#140

Post by inciampati »

saltcured wrote: post #103
intellectronica wrote: post #11

I love the title "Big LLMs" because it means that we are now making a distinction between big LLMs and minute LLMs and maybe medium LLMs. I'd like to propose the we call them "Tall LLMs", "Grande LLMs", and "Venti LLMs" just to be precise.

I'd prefer to see olive sizes get a renaissance. I was always amused by Super Colossal when following my mom around a store as a little kid. From a random web search, it seems the sizes above Large are: Extra Large, Jumbo, Extra Jumbo, Giant, Colossal, Super Colossal, Mammoth, Super Mammoth, Atlas.

And I'd love to see data compression terminology get an overhaul. Do we need big LLMs or just succinct data structures? Or maybe "compact" would be good enough? (Yeah LLMs are cool but why not just, you know, losslessly compress the actual data in a way that lets us query its content?)
Post reply on HN