Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Billion-Parameter Theories

worldgov.org

Post reply on HN
Print view View original

81–90 of 92 posts

Re: Billion-Parameter Theories

#81

Post by alex-moon »

wavemode wrote: post #58

Not to sound condescending, but this reads like someone fimiliar with LLMs but very unfamiliar with statistics in general. If we could understand economics, or poverty, or any number of other social structures, simply by cramming data into a statistical model with billions of parameters, we would've done that decades ago and these problems would already be understood. In the real world, though, there is a phenomenon…

I think this is a really important distinction to make. The OP seems to be making a fallacious equivocation on the word "parameter" - specifically, any individual "parameter" in a large ML model has no unit of measurement because it doesn't mean anything on its own. I watched a great documentary about the "Soft Hair on Black Holes" paper where they talk about having to move from the blackboard to the computer because the equation explodes into thousand of parameters - the key thing to understand being that each of those parameters represents some "real" thing, a momentum, a charge, a curvature, etc.

Re: Billion-Parameter Theories

#82

Post by squeefers »

roughly wrote: post #51

There's a lot of ink in this spent on how Poverty, Climate Change, Urban Decay, and Financial Markets are Complex Hard Complicated problems. The problem with these is they're also problems where there are actors profiting from the failure to fix the system - the issue isn't that we don't understand the complex nature of the domain, it's that the components of the system actively and agentically resist changes to the…

[dead]

Re: Billion-Parameter Theories

#83

Post by pixl97 »

zkmon wrote: post #47
pixl97 wrote: post #39

Earlier quoted context omitted.

Right, dinosaurs were perfectly fine, their ignorance worked out well for them. The universe is uncaring, simply not giving a shit if you have knowledge or not. Knowledge gives you the ability to survive minor conniption fits of cosmic magnitude, and at the same time gives you a gun to shoot your own foot off. There ain't no such thing as a free lunch.

So you think your tech can help you survive the event that made dinosaurs go bust.

Yep, given enough forewarning with current technology asteroid redirection isn't out of the question.

Then again, we'll likely get ourselves with global warming first.

Re: Billion-Parameter Theories

#84

Post by red-iron-pine »

newyankee wrote: post #53
roughly wrote: post #51

There's a lot of ink in this spent on how Poverty, Climate Change, Urban Decay, and Financial Markets are Complex Hard Complicated problems. The problem with these is they're also problems where there are actors profiting from the failure to fix the system - the issue isn't that we don't understand the complex nature of the domain, it's that the components of the system actively and agentically resist changes to the…

Literally countries with so much surplus land: Canada, Australia etc. have housing crisis where most of the top 10-20% of the population has become speculators in housing and openly NIMBY with no interest in supply side solutions unless forced down.

most of the surplus land is marginally habitable, and costs increase dramatically when you get rural.

plus "land" doesn't mean anything if you're not near the people and things you want do to, places to work, etc.

do you want to do a 1.5 hour commute and hustle to live around Toronto, or do you want to live in Outer Nowhere, Manitoba, population 400, and where it regularly gets to -40C?

Re: Billion-Parameter Theories

#85

Post by ihumanable »

In the author's own analogy of blacksmithing and metallurgy, I see an interesting parallel.

Humans worked metal for a long time and you can make better and better forges without knowing the metallurgy of why the result is better. If I make the fire hotter the metal comes out better, and I can get to work making forges that produce hotter and hotter fire.

LLMs could in this analogy be the forge. We can make them bigger and bigger and get better and better answers out, in the same way a pre-metallurgy human could make their forges hotter and hotter and get better and better metal out.

But the hottest forge doesn't mean you get metallurgy.

Re: Billion-Parameter Theories

#86

Post by red-iron-pine »

strken wrote: post #55
newyankee wrote: post #53

Earlier quoted context omitted.

Literally countries with so much surplus land: Canada, Australia etc. have housing crisis where most of the top 10-20% of the population has become speculators in housing and openly NIMBY with no interest in supply side solutions unless forced down.

The problem we've got is that 10-20% of the population are speculating while another 50% of the population have almost their entire net worth stuffed into their family home. We're finding it difficult to rein in the top without ruining the middle too.

there are plenty of examples of the top getting reined in, and if people have their wealth in their house then don't take their houses.

this isn't a complicated problem, and it's not difficult to build guillotines

Re: Billion-Parameter Theories

#87

Post by DoctorOetker »

pdonis wrote: post #76
DoctorOetker wrote: post #73

Earlier quoted context omitted.

This. Every sedentary society has historically scared its members of the dangers of the nomadic lifestyle, heathens, ... The implied conclusion being that since our ancestors switched from nomadic to sedentary it must have been preferable, a kind of informal democratic collectively and individually approved choice. Surely sedentary must have been better, how else could such a transition have been sustained? Rather ea…

If you think a subsistence nomadic lifestyle is preferable to a modern "sedentary" one, then how are you able to post here? Subsistence nomads don't have Internet access (to name just one of umpteen things we "sedentary" moderns have access to that they don't). There are ways to live off grid if you really think it's preferable.

I am a homeless bum, I literally live under a bridge.

Re: Billion-Parameter Theories

#88

Post by DoctorOetker »

pdonis wrote: post #77
DoctorOetker wrote: post #73

Earlier quoted context omitted.

This. Every sedentary society has historically scared its members of the dangers of the nomadic lifestyle, heathens, ... The implied conclusion being that since our ancestors switched from nomadic to sedentary it must have been preferable, a kind of informal democratic collectively and individually approved choice. Surely sedentary must have been better, how else could such a transition have been sustained? Rather ea…

> even if you escape the sedentary village, the stable neighbouring village will happily return you to "your owner" so that he would hopefully return the favor if ever he catches one of "their slaves") Tell that to all the people who ran the Underground Railroad in the pre-Civil War US, not to mention all the other ways that Fugitive Slave laws were persistently violated. I think you are vastly underestimating the be…

What makes you think they don't post on the internet?

Re: Billion-Parameter Theories

#89

Post by pdonis »

DoctorOetker wrote: post #88
pdonis wrote: post #77

Earlier quoted context omitted.

> even if you escape the sedentary village, the stable neighbouring village will happily return you to "your owner" so that he would hopefully return the favor if ever he catches one of "their slaves") Tell that to all the people who ran the Underground Railroad in the pre-Civil War US, not to mention all the other ways that Fugitive Slave laws were persistently violated. I think you are vastly underestimating the be…

What makes you think they don't post on the internet?

How is someone living a hunter-gatherer subsistence life going to get Internet access? That requires a technological society, which requires a lot of wealth creation way above a subsistence level.

If you're saying that someone might claim they're living a hunter-gatherer subsistence life except when they're not, well, that's just hypocrisy. If you're going to make use of things that require a modern technological society, then you're saying life in a modern technological society is preferable to a hunter-gatherer subsistence life, whether you like it or not. You can't have it both ways.

Re: Billion-Parameter Theories

#90

Post by pdonis »

DoctorOetker wrote: post #87
pdonis wrote: post #76

Earlier quoted context omitted.

If you think a subsistence nomadic lifestyle is preferable to a modern "sedentary" one, then how are you able to post here? Subsistence nomads don't have Internet access (to name just one of umpteen things we "sedentary" moderns have access to that they don't). There are ways to live off grid if you really think it's preferable.

I am a homeless bum, I literally live under a bridge.

Fine. And whatever device you're using to post here just happened to emerge spontaneously from the dirt, instead of being built by the efforts of thousands of people spread all over the world as part of a modern technological society.

Also: where do you get your food? Do you grow it? Or hunt for it in a natural wilderness, untouched by technology, using tools you made yourself, without the benefit of modern technology?

Where do you get your clothes? Do you make them yourself? Out of natural materials that would be there if our modern, technological society did not exist?

I'm going to make a wild guess that the answers to those questions are "no"--that you are relying on sources of food and clothes that also require a modern technological society. Not to mention transportation and whatever else you need to do the things that occupy your day.

So no, you are not living a hunter-gatherer subsistence life. You are taking advantage of the fact that it is possible in a modern technological society to be a homeless bum living under a bridge, without having to do all the things that actual hunter-gatherers living a subsistence life have had to do all through human history to survive.

Post reply on HN