Live data from Hacker News

Many in the AI field think the bigger-is-better approach is running out of road

economist.com

271–280 of 354 posts

Re: Many in the AI field think the bigger-is-better approach is running out of road

#271
post #261

Earlier quoted context omitted.

And you don't have a shred of actual evidence to demonstrate that, only your own preconceptions about how things supposedly are.

The burden is on you to prove your claims.

I'm not trying to demonstrate that my claims are true. I'm trying to demonstrate that it is meaningless to discuss these topics in the first place, because we don't understand the workings of the mind nearly well enough to distinguish things like "truthfulness" and "concepts".

Re: Many in the AI field think the bigger-is-better approach is running out of road

#272

Earlier quoted context omitted.

We're able to do something analogous to reinforcement learning (take on new example data to update our 'weights'). Why do I spend time debating these ideas on Hacker News? Probably the underlying motivation is improving the reliability of my model of the world, which over my lifetime and the lifetimes of creatures before me has led to (somewhat indirectly) positive outcomes in survival and reproduction. Is my model o…

Is my model of the world that different to that of an LLM? Well, one major way you’re different from an LLM is that you’re alive. You’re capable of learning continuously as you go about your day and interact with the world. LLMs are “dead” in the sense that they’re trained once and frozen, to be used from then on in the exact same state of their initial training.

I agree that is a fundamental difference. That’s what I meant about reinforcement learning. Our ‘model weights’ are being updated with new data all the time.

I was just referring to what happens at a specific instance in time when someone asks me for example ‘What’s the capital of Norway?’

Re: Many in the AI field think the bigger-is-better approach is running out of road

#273
post #271

Earlier quoted context omitted.

The burden is on you to prove your claims.

I'm not trying to demonstrate that my claims are true. I'm trying to demonstrate that it is meaningless to discuss these topics in the first place, because we don't understand the workings of the mind nearly well enough to distinguish things like "truthfulness" and "concepts".

Not we. you. Do not foist your ignorance on others. I recommend foundations of neuroscience by Henley.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#274
post #257
post #235

Earlier quoted context omitted.

> It is undeniable that human reasoning is a stochastic process It can still be a deterministic process. If anything came out of the whole LLM story for me it is that I am even more convinced that it is. My (somewhat educated, but still naive) idea why it looks like a stochastic process is that the brain gets incredible amounts of random input. We literally get bombarded with particles and energy ever instance we liv…

According to physics, it must be stochastic. The only question is distribution, and whether we can ignore outlier cases.

Are you saying this because quantum mechanics is random in some interpretations and is our most fundamental theory?

That’s a bit of a stretch when we don’t actually know if QM is objectively random, but it could be sure. But then what about things like, is it random that 1+1=2. No…what are you really saying is random when a human answers this? I think even if you make the assumption QM is objectively random, thought is the hard problem after all and we might not want to jump so far head. Math certainly isn’t random and we can think about it.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#276
post #155

Earlier quoted context omitted.

> A LM doesn't understand "truthfulness". It has no concept of a sequence being true or not, only of a sequence being probable. I claim that the human brain doesn't understand "truthfulness" either. It merely creates the impression that understanding is taking place, by adapting to social and environmental pressures. The brain has no "concepts" at all, it just generates output based on its input, its internal wiring,…

> Do you have any evidence that contradicts that claim? Empirical evidence? Yes I do. The brain commands an entity that has to exist and function in the context of objective reality. Being unable to verify it's internal state against that, would have been negatively selected some time ago, because stating: "I'm sure that rumbling cave bear with those big sharp teeth is a peaceful herbivore" won't change the objective…

I think this is incomplete on a number of levels. For a start, to be interesting, “truth” has to be something than just whatever your eyes can see. There have been wars (culture, economic, kinetic, etc) fought to define something as a truth.

The concept of truth is notoriously hard for humans to grapple with. How do we know something is true isn’t just a neurobiological question, it’s been grappled with throughout the history of philosophy — including major revisions of our understanding in the past 80 years.

And for the record, rumbling cave bears are mostly peaceful herbivores.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#277
post #11

We need a way to make tight little specialist models that don't hallucinate and reliably report when they don't know. Trying to cram all of the web into a LLM is a dead end.

I'm trying to get funding to do just this. Essentially nested LLMs were every LLM is very topical and if it's not trained on something there's indexes to other LLMs it can ask to get the right answer. I think of it kinda like a hub and spoke model.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#278

Earlier quoted context omitted.

Punishment is not effective for most people in reducing crime. One thing that is effective is increasing the perceived risk of getting caught. People rarely commit crimes when they are sure they will be caught.

Even in the worst case, recidivism rates only reach around 2/3. That means punishment (incarceration) reforms criminals 1/3 the time. When you consider all of the problems with prisons in the U.S., and all of the hurdles offenders have to pass in order to successfully reintegrate into society, it’s quite remarkable that so many manage to do so. There are plenty of medical interventions, particularly for mental illnes…

Correlation is not causation. If punishment is causing the recidivism then you'd expect an increase in punishment to increase recidivism. It doesn't. You also need to compare prevention with recidivism. Optimal approaches prevent the crime in the first place.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#279
post #208

Earlier quoted context omitted.

Because the punishment is part of the input that determines what you will do. In fact, if free will were absolute, punishment wouldn't make any sense because it wouldn't have any effect on your will.

Neither of your claims make sense. If everything is predetermined, you don't need inputs. If you have absolute free will you are free to disregard or consider inputs.

By that logic, a deterministic computer program wouldn’t need inputs.

What is being claimed by a deterministic world model is that the output (behavior and internal state change) of a human is a pure function of its current state and inputs. Then we try to give inputs that will lead to desired outputs.

The non-compatibilist view of free will is that it is not a pure function, namely that there is a third independent factor, the “free will”, that influences the behavior (and possibly the internal state). If that is the case, there may never be a way to choose inputs that lead to the desired outputs, because the free will could simply void their effect.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#280
post #246

Earlier quoted context omitted.

1) Computers, smart phones, and pocket calculators also outperform humans at mental tasks. So do birds, dolphins, and dogs for that matter, at tasks for which they are specialized. 2) so? What are you imagining this implies? An infinity of possibilities does not a reason make, unless you are talking about arbitrary religious beliefs. 3) Right, no goals, no will, no purpose. Just some matrix multiplies doing interesti…

It seems a safe prediction if you extrapolate that AI will get generally smarter than humans. It also seems a safe prediction, given past human behaviour that some humans will set some AI to do bad stuff. Therefore risk. (eg "chat gtp 27, help me make billions on crypto and use it to set up a distributed army to take over the world")

> chat gtp 27, help me make billions on crypto and use it to set up a distributed army to take over the world

Yevgeny Prigozhin has entered the chat.

Post reply on HN