Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Will scaling work?

dwarkeshpatel.com

Post reply on HN
Print view View original

281–289 of 289 posts

Re: Will scaling work?

#281

Post by mgaunard »

ryanklee wrote: post #113
mgaunard wrote: post #37

I think the more interesting question is how long will people cling to the illusion that LLMs will lead us to AGI? Maintaining the illusion is important to keep the money flowing in.

You say this is as if it's settled and obvious that it it's an illusion and it's only the delusional that believe the opposite. But if it were so settled and obvious there would be a clear line of reasoning to make that plain. And there is not. Instead, there is a very vibrant debate on the topic with tons of nuance and good faith (and bad) on each side, if we want to talk about sides. And, of course, one of the impl…

There are tens of billions in the balance, how can you make sure the arguments are made in good faith? Money is self-reinforcing as well.

Re: Will scaling work?

#282

Post by ryanklee »

mgaunard wrote: post #281
ryanklee wrote: post #113

Earlier quoted context omitted.

You say this is as if it's settled and obvious that it it's an illusion and it's only the delusional that believe the opposite. But if it were so settled and obvious there would be a clear line of reasoning to make that plain. And there is not. Instead, there is a very vibrant debate on the topic with tons of nuance and good faith (and bad) on each side, if we want to talk about sides. And, of course, one of the impl…

There are tens of billions in the balance, how can you make sure the arguments are made in good faith? Money is self-reinforcing as well.

> how can you make sure the arguments are made in good faith

Not sure why I would need to. I'm general, arguments can simply be understood. If they are good, good. If not, then fix them or move along.

Good and bad faith are about people, not arguments.

Re: Will scaling work?

#283

Post by herval »

jodrellblank wrote: post #280
herval wrote: post #279

Earlier quoted context omitted.

that's quite a restrictive view of how much the internet changed _everything_ across the planet, from culture to work

Oh what a frustratingly low effort reply to my high effort comment. Come on, if you want to make the claim that the internet changed _everything_ you can back it up with some examples - especially when the claim was not "nothing changed" but "things don't feel as different" - so the examples should not be "someone far away does something differently" or "some behind the scenes organisation was more efficient", but th…

True, I’m not very used to internet debates lasting longer than one or two low effort replies, so I became dismissive too. Apologies.

I don’t really have hard data that proves the impact of the internet at a macro level, just anecdotal stuff. I think it’s easy to overplay the impact of past innovations bc we see the late waves - eg the telephone wasn’t really a thing that impacted most of the world for decades, and there’s entirely countries that went from no telephone at all to whatsapp directly (brazil for instance)

When it comes to jobs, I think I know maybe a dozen people whose jobs/industries only exist because of the Internet - and that’s counting outside the tech bubble, mind you. I’m sure this is verifiable with labor data somehow. Not to mention the entire middle class slipping into gigs in a model that only exists because of the internet (maybe a bit too recent for most to grasp the consequences).

On a personal level - quite literally everything in my life would’ve been completely different. I can’t even imagine the kind of local job I’d have at a small town, watching about stuff on the TV and hearing about tech only from a friend over radio

Re: Will scaling work?

#284

Post by lern_too_spel »

elktown wrote: post #277
lern_too_spel wrote: post #275

Earlier quoted context omitted.

Why did you criticize the comment if you don't think it deserves criticism? Words have meaning.

> No, not at all. The only thing I did was to react on how ridiculous that oversimplification is and how such a thing can only come about due to an embarrassing amount of hubris currently going around our field with relation to "AI". It's a hand-wavy "Eh, how hard can it be?" comment to rationalize ML being a pathway to AGI. Added emphasis, and others reacted the same way. You can disagree with the interpretation but…

Visarga's comment is equivalent to comments that neuroscientists make, as I showed earlier, so inferring that it means, "How hard can it be?" is your mistake alone, not visarga's. The voters agree.

Re: Will scaling work?

#285

Post by elktown »

lern_too_spel wrote: post #284
elktown wrote: post #277

Earlier quoted context omitted.

> No, not at all. The only thing I did was to react on how ridiculous that oversimplification is and how such a thing can only come about due to an embarrassing amount of hubris currently going around our field with relation to "AI". It's a hand-wavy "Eh, how hard can it be?" comment to rationalize ML being a pathway to AGI. Added emphasis, and others reacted the same way. You can disagree with the interpretation but…

Visarga's comment is equivalent to comments that neuroscientists make, as I showed earlier, so inferring that it means, "How hard can it be?" is your mistake alone, not visarga's. The voters agree.

> as I showed earlier

you mean by picking the intro one-liner from a textbook? haha, pathetic. whatever floats your boat mate.

Re: Will scaling work?

#286

Post by lern_too_spel »

elktown wrote: post #285
lern_too_spel wrote: post #284

Earlier quoted context omitted.

Visarga's comment is equivalent to comments that neuroscientists make, as I showed earlier, so inferring that it means, "How hard can it be?" is your mistake alone, not visarga's. The voters agree.

> as I showed earlier you mean by picking the intro one-liner from a textbook? haha, pathetic. whatever floats your boat mate.

You picked an intro one-liner from visarga's comment. It's amazing that you really can't see that this is the same thing and yet are capable of using the Internet. Neither one claims to know how the brain does what it does, but they both say that the brain does the same thing.

Re: Will scaling work?

#287

Post by elktown »

lern_too_spel wrote: post #286
elktown wrote: post #285

Earlier quoted context omitted.

> as I showed earlier you mean by picking the intro one-liner from a textbook? haha, pathetic. whatever floats your boat mate.

You picked an intro one-liner from visarga's comment. It's amazing that you really can't see that this is the same thing and yet are capable of using the Internet. Neither one claims to know how the brain does what it does, but they both say that the brain does the same thing.

I picked the line within a context, explained the context multiple times to you to make sure. You picked the line, removed all context of the discussion and the goes on to compare it a randomly picked textbook intro line. Simply mindbogglingly stupid.

Re: Will scaling work?

#288

Post by crowbahr »

tomduncalf wrote: post #261
crowbahr wrote: post #76

Earlier quoted context omitted.

To say there's no magic in the brain drastically *minimizes the complexity of the brain. Your brain is several orders of magnitude more complex than even the largest LLM. GPT4 has 1 trillion parameters? Big deal. Your brain has 1 quadrillion synapses, constantly shifting. Beyond that the synapses are analog messages, not binary. Each synapse is approximately like 1000 transistors based on the granularity of messaging…

Re: synapses being analog messages, isn’t this sort of true of neural networks? In my understanding, the weights, biases and values flowing through the network are floating point numbers so I’d argue closer to analog than binary.

You know that's a good point that I didn't consider in my comment. LLMs use 16bit floats, which is approximately the same number of values.

But then you get into the nuance of spiking vs non spiking neurons etc, which to my knowledge isn't emulated. The brain is extremely complex. I don't believe LLMs do anything similar to inhibitor neuronal function for example.

Re: Will scaling work?

#289

Post by lern_too_spel »

elktown wrote: post #287
lern_too_spel wrote: post #286

Earlier quoted context omitted.

You picked an intro one-liner from visarga's comment. It's amazing that you really can't see that this is the same thing and yet are capable of using the Internet. Neither one claims to know how the brain does what it does, but they both say that the brain does the same thing.

I picked the line within a context, explained the context multiple times to you to make sure. You picked the line, removed all context of the discussion and the goes on to compare it a randomly picked textbook intro line. Simply mindbogglingly stupid.

Not randomly picked. The introduction. Visarga's comment is not a treatise on how exactly to reproduce the brain. The fact that you still don't understand this is seriously impressive.
Post reply on HN