Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Many in the AI field think the bigger-is-better approach is running out of road

economist.com

Post reply on HN
Print view View original

261–270 of 354 posts

Re: Many in the AI field think the bigger-is-better approach is running out of road

#261

Post by p-e-w »

endisneigh wrote: post #251
p-e-w wrote: post #155

Earlier quoted context omitted.

> A LM doesn't understand "truthfulness". It has no concept of a sequence being true or not, only of a sequence being probable. I claim that the human brain doesn't understand "truthfulness" either. It merely creates the impression that understanding is taking place, by adapting to social and environmental pressures. The brain has no "concepts" at all, it just generates output based on its input, its internal wiring,…

I find these takes so lazy. What you have claimed here is just totally wrong.

And you don't have a shred of actual evidence to demonstrate that, only your own preconceptions about how things supposedly are.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#262

Post by mensetmanusman »

bemmu wrote: post #111
michaelteter wrote: post #63

Isn't the fundamental problem that LLM's don't actually understand anything (as greater concepts), but rather operate as complex probability machines? My 2 month active experience with ChatGPT-4 gave me the following takeaways: - when it's right, it's amazing; and when you, the operator, can recognize the niche use case where it performs really well, it can be a game-changer (although you could have programmed a tool…

I’m not convinced the language part of my brain isn’t just a complex probability machine, just with different trade-offs.

That just means you don't believe in free will, which is fine. This also means you had no choice to make this comment otherwise :)

Re: Many in the AI field think the bigger-is-better approach is running out of road

#263

Post by rco8786 »

Animats wrote: post #135
rco8786 wrote: post #19

Earlier quoted context omitted.

Bingo. I've been beating this drum since the initial GPT-3 awe.. The future of AI is bespoke, purpose-driven models trained on a combination of public and (importantly) proprietary data. Data is still king.

You need sort of a "primary education" data set which gets the model up to roughly a high school education level. Then augment it with special-purpose models for specific areas. But until "I don't know" comes out, rather than hallucinations, we're in trouble.

> You need sort of a "primary education" data set which gets the model up to roughly a high school education level.

Yea that was what I was getting at with the "combination" of data. The publicly available data provides the base/primary education, then you specialize it with your proprietary data and bam, you have an AI model that nobody else can produce...an actual product moat.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#264

Post by mensetmanusman »

Legend2440 wrote: post #122
anikan_vader wrote: post #117

Earlier quoted context omitted.

There is plenty of evidence for non-determinism in matter, which the brain is notably made out of.

Not necessarily. Everything is deterministic above the quantum level, and it's possible that quantum non-determinism is the result of deterministic processes we can't see. Lots of deterministic processes (like PRNGs) look random from the outside - that's what chaos theory is about. I think it's likely that everything in the universe is deterministic.

Not necessarily. As you fine grain the simulation enough, eventually you run out of energy in the universe to compute even a second of simulated output.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#265

Post by mensetmanusman »

trashtester wrote: post #245
tchaffee wrote: post #175

Earlier quoted context omitted.

Why would you need to justify a punishment that was already predetermined? If you are going to excuse crime with the no free will argument you can excuse the punishers with that argument too.

I'm in favor of punishing crimes. I don't think I have a choice about the matter.

Prisoners don't have free will, nor do judges and police. All is well!

Re: Many in the AI field think the bigger-is-better approach is running out of road

#266

Post by [deleted] »

layer8 wrote: post #209
Legend2440 wrote: post #162

Earlier quoted context omitted.

Maybe, but there's no data to support one interpretation over another. I believe that all quantum interpretations are incomplete and therefore wrong. Quantum mechanics is an abstraction over a deeper level of physics we can't measure yet.

Objective-collapse theories can actually be experimentally tested: https://en.wikipedia.org/wiki/Objective-collapse_theory#Test...

[deleted]

Re: Many in the AI field think the bigger-is-better approach is running out of road

#267

Post by p-e-w »

b1daly wrote: post #240
p-e-w wrote: post #167

Earlier quoted context omitted.

> Show me any evidence for AI risk today beyond people's theories and beliefs? Deduction. Empirical evidence isn't the only source of insight. You don't have to conduct experiments in order to reasonably conclude that an entity that 1. outperforms humans at mental tasks 2. shares no evolutionary commonality with humans 3. does not necessarily have any goals that align with those of humans is a potential threat to hum…

the world is incredibly filled with risk to humans—people in the AI doomer camp are making a claim that AI potentially is a new kind of uncontrollable risk that warrants extraordinary regulation the basis of this claim seems to be a confusion of logical or deductive reasoning with inductive or observational reasoning argument comes down to - it’s possible to imagine a super intelligent machine that has properties tha…

> since it’s possible to imagine it, this means it will come into existence

Nope. That's not the argument. In fact, it's such a bad take that it reeks of a deliberately constructed strawman.

The actual argument is: Since it's possible to imagine it, and doesn't contradict any known laws of nature or technology, and current development appears to be iterating towards it, it might come into existence, thus it presents a statistical risk.

When I take out tornado insurance, it's not because I know my house will be blown away by a storm – it's because I don't know, but the possibility is there.

Certainty is not required in order to conclude that risk exists. Quite the opposite is true: Risk is a function of uncertainty.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#268

Post by krisoft »

elif wrote: post #254

After a year of Tesla FSD beta I agree 100%. I used to think that adding more and more and more real world video events to the training pool was contributing at least a diminishing value to the model. But the worst of all behaviors have not diminished and some have actually gotten worse. Most of the improvements now come through what feels like manual heuristics and tuning parameters rather than any actual improvemen…

> Most of the improvements now come through what feels like manual heuristics and tuning parameters rather than any actual improvement in intelligence.

So if I understand your are driving a Tesla car with Full Self-Driving (FSD) capability, but you are not an engineer at Tesla who is privy to implementation details.

How do you know if a change you perceive is caused by model retraining vs manual heuristics and tuning parameters?

Re: Many in the AI field think the bigger-is-better approach is running out of road

#269

Post by endisneigh »

p-e-w wrote: post #261
endisneigh wrote: post #251

Earlier quoted context omitted.

I find these takes so lazy. What you have claimed here is just totally wrong.

And you don't have a shred of actual evidence to demonstrate that, only your own preconceptions about how things supposedly are.

The burden is on you to prove your claims.

Re: Many in the AI field think the bigger-is-better approach is running out of road

#270

Post by tchaffee »

layer8 wrote: post #208
tchaffee wrote: post #179

Earlier quoted context omitted.

If you don't have free will then how would something discourage you from doing something that is already determined you will do? That doesn't make any sense.

Because the punishment is part of the input that determines what you will do. In fact, if free will were absolute, punishment wouldn't make any sense because it wouldn't have any effect on your will.

Neither of your claims make sense. If everything is predetermined, you don't need inputs.

If you have absolute free will you are free to disregard or consider inputs.

Post reply on HN