Are there any search experiences that allow me to search like it's 1999? I'd love to be able to re-create the experience of finding random passion project blogs that give a small snapshot of things people and business were using the web for back then.
Big LLMs weights are a piece of history
201–210 of 222 posts
Re: Big LLMs weights are a piece of history
#202“We should regard the Internet Archive as one of the most valuable pieces of modern history; instead, many companies and entities make the chances of the Archive to survive, and accumulate what otherwise will be lost, harder and harder. I understand that the Archive headquarters are located in what used to be a church: well, there is no better way to think of it than as a sacred place.” Amen. There is an active effor…
Anyone who takes even an hour to audit anything about the Internet Archive will soon come to a very sad conclusion. The physical assets are stored in the blast radius of an oil refinery. They don't have air conditioning. Take the tour and they tell you the site runs slower on hot days. Great mission, but atrociously managed. Under attack for a number of reasons, mostly absurd. But a few are painfully valid.
EDIT: asking Claude:
Based on historical data, major refinery explosions in developed countries might occur at a rate of approximately 1 in 1,000 to 1 in 2,000 refinery-years of operation. Using this very rough estimate, a single refinery might have approximately a 50% chance of experiencing a significant explosion somewhere between 700-1,400 years of continuous operation.
Re: Big LLMs weights are a piece of history
#203Earlier quoted context omitted.
Their yearly budget is less than the budget of just the SF library system.
What I don't understand is how do they afford the storage costs? Surely it must be pricey to have 70+ petabytes of data that's only growing.
A big chunk was outsourced to Sun at one point. And that name alone should tell you how current the information is. https://en.wikipedia.org/wiki/Sun_Modular_Datacenter
In 2020 at least one public filing shows expenses of $19.9MM with $9.2MM classified as wages. So no more than $900k/month in 2020 and maybe double that now. Recent data is messy due to Covid donations and lawsuits.
Re: Big LLMs weights are a piece of history
#204Earlier quoted context omitted.
Anyone who takes even an hour to audit anything about the Internet Archive will soon come to a very sad conclusion. The physical assets are stored in the blast radius of an oil refinery. They don't have air conditioning. Take the tour and they tell you the site runs slower on hot days. Great mission, but atrociously managed. Under attack for a number of reasons, mostly absurd. But a few are painfully valid.
How significant is "in the blast radius of an oil refinery"? Once every how many years should I expect a typical oil refinery to explode? This really doesn't seem like it should be their first, second, fifth, or twelfth priority to"solve". EDIT: asking Claude: Based on historical data, major refinery explosions in developed countries might occur at a rate of approximately 1 in 1,000 to 1 in 2,000 refinery-years of op…
https://apen4ej.medium.com/11-years-ago-the-chevron-refinery...
Keep in mind that Brewster bought the building because it looked like the icon, not vice versa. Not exactly the amount of thought that might be expected of an archival institution.
Re: Big LLMs weights are a piece of history
#205“We should regard the Internet Archive as one of the most valuable pieces of modern history; instead, many companies and entities make the chances of the Archive to survive, and accumulate what otherwise will be lost, harder and harder. I understand that the Archive headquarters are located in what used to be a church: well, there is no better way to think of it than as a sacred place.” Amen. There is an active effor…
Anyone who takes even an hour to audit anything about the Internet Archive will soon come to a very sad conclusion. The physical assets are stored in the blast radius of an oil refinery. They don't have air conditioning. Take the tour and they tell you the site runs slower on hot days. Great mission, but atrociously managed. Under attack for a number of reasons, mostly absurd. But a few are painfully valid.
Re: Big LLMs weights are a piece of history
#206Earlier quoted context omitted.
Anyone who takes even an hour to audit anything about the Internet Archive will soon come to a very sad conclusion. The physical assets are stored in the blast radius of an oil refinery. They don't have air conditioning. Take the tour and they tell you the site runs slower on hot days. Great mission, but atrociously managed. Under attack for a number of reasons, mostly absurd. But a few are painfully valid.
There should not be any physical centralization. Use a series of redundant IPFS pins and/or torrents or some decentralized database of some kind.
Also until recently their whole model was storing physical material (on an active fault line next to an oil refinery) then allowing digital access to it. Courts ruled that illegal for modern works.
Re: Big LLMs weights are a piece of history
#207It might be possible to create an L LM that can write a custom vintage game or program on demand in machine code and simultaneously generate assets like sprites. Especially if you use the latest reinforcement learning techniques.
Re: Big LLMs weights are a piece of history
#208Earlier quoted context omitted.
It's a crazy experience being just physically larger than most of the world. Especially when the size on the label carries some implicit shame/judgement. Like I'm skinny, I'm pretty much the lowest weight I can be and not look emaciated / worrying. But when shopping for a skirt in Asian sizes I was a 4XL, and usually an or L-2XL in European sizes. Having to shift my mental space that a US M is the "right" size for me…
The shame you feel is yours, it's not inherent to the sizing.
And my experience isn't unique in any way here and it's really hard to not see it pervasive through our culture.
Re: Big LLMs weights are a piece of history
#209Earlier quoted context omitted.
Well, it did establish a new HQ in Canada… https://vancouversun.com/news/local-news/the-internet-archiv... (Edited: apparently just a new HQ and not THE HQ)
With this belligerent maniac in the White House who recently doubled-down on his wish to annex Canada [1], I wouldn't feel safe relocating there if the goal is to flee the US. [1] https://www.nbcnews.com/politics/donald-trump/trump-quest-co...
Re: Big LLMs weights are a piece of history
#210Earlier quoted context omitted.
To continue my earlier comment... I prefer not to call an LLM "intelligent" much less "outrageously intelligent". Why? The main reason is communication clarity -- and by communication I mean the notion of a sender communicating a meaning to a receiver. Not just symbolic information (a la Shannon), but a faithful representation in the recipient. The phrase "outrageously intelligent" can have many conflicting interpret…
Intelligence is closely related to the concept of attractiveness and gravitas. You say it depends on the agent. I say it's in the eye of the beholder. People aren't very good at explaining what attracts them either. The closest thing we have to a definition for intelligence is probably the LLMs themselves. They're very good at predicting words that attract people. So clearly we've figured it out. It's just such a sha…
Interesting. I wonder why you make this connection. Do you know?
Your choice of definition seems to be what I would call "perception of intelligence". But why add that extra layer of indirection; why require an observer? I claim this extra level of indirection is not necessary. I eschew definitions with unnecessary complexity (a.k.a "accidental complexity" in the phrasing of Rich Hickey).
Here are some examples that might reveal problems with the definition above:
- DeepBlue (decisively beating Kasparov in 1997) showed a high level of intelligence in the game of chess. The notion of "being good at the game" is simpler (conceptually) than the notion of "being attractive to people who like the game of chess". See what I mean?
- A group of Somali pirates working together may show impressive tactical abilities, including the ability to raid larger ships, which I would be willing to call a form of tactical intelligence to achieve their goals. I grant the intelligent behavior even though I don't find it "attractive", nor do I think the pirates need any level of "gravitas" to do it. Sure, the pirates might use leadership, persuasion, and coordination to accomplish their goals but these concepts are a means to an end accomplishing the goal. But these traits are not necessary. Since intelligent behavior can be defined without using those concepts, why include them? Why pin them to the definition?
- The human brain is widely regarded as a intelligent organ in a wide variety of contexts relating to human survival. Whether or not I find it "attractive" is irrelevant w.r.t. intelligence, I say. If the neighboring tribe wants to kill me and my tribe (using their tribally-oriented brains), I would hardly call their brains attractive or their methods being nuanced enough to use "gravitas".
My claim is then: Intelligence should be defined by functional capability which leads to effectiveness at achieving goals, not by how we feel about the intelligence or those displaying it.