Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018) • Hacker News Board

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#61

Post by jw1224 » Fri, Oct 28, 2022, 12:23 PM UTC

Galaxeblaffer wrote: post #58

Randomly watched this yesterday https://www.youtube.com/watch?v=hXgqik6HXc0&ab_channel=LexFr... where Roger Penrose argues that we're missing something fundamental about consciousness and his best bet is a structure called the microtubules. This talk reminded me of my own research into "AI" back in the 00's and that it's almost impossible to talk about AI since everybody has a different idea as to whay AI is, yes i k…

Roger Penrose & Stuart Hammeroff’s “Orchestrated Objective Reduction” theory [1] is fascinating and really captured my imagination when I came across it.

But like almost all scientific theories tackling The Hard Problem, it’s built on the assumption that matter gives rise to consciousness.

As time goes by and my own understanding deepens, I’m becoming more and more convinced that this assumption is wrong. Instead we should start considering that consciousness is fundamental, and matter is a product of universal conscious experience.

Idealism is still compatible with the material world, but it seems futile to search for “the experiencer” within the experience itself.

[1] https://en.m.wikipedia.org/wiki/Orchestrated_objective_reduc...

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#62

Post by kordlessagain » Fri, Oct 28, 2022, 12:23 PM UTC

Galaxeblaffer wrote: post #58

Randomly watched this yesterday https://www.youtube.com/watch?v=hXgqik6HXc0&ab_channel=LexFr... where Roger Penrose argues that we're missing something fundamental about consciousness and his best bet is a structure called the microtubules. This talk reminded me of my own research into "AI" back in the 00's and that it's almost impossible to talk about AI since everybody has a different idea as to whay AI is, yes i k…

Having watched Penrose and Hammeroff for a while now, IF microtubules contribute to consciousness experience then it is the collapse (or inference) that gives it wheels.

I'm laughing here because when I posted their ideas to HN oh so long ago, I got downvoted to oblivion because "there's no way organic matter can act as a quantum device". For a place that considers itself a "safe" place to explore ideas, it can be quite dangerous to share too much too early, sometimes.

Time will tell, but my instincts are that we're getting close. We needed computers dreaming first, and we have that now with generative networks!

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#63

Post by solsane » Fri, Oct 28, 2022, 12:57 PM UTC

data_maan wrote: post #34

Animats wrote: post #6

This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…

> Current hardware is easily up to the task. I don't think so. If you want to model a single synapse in full to capture all effects that might lead to "learning", you have a system of ordinary differential equations. Solving that is very hard, and solving that for 10 million neurons is impossible. On current hardware can only implement but a poor caricature of a real neuron.

Real neurons are far slower (interaction is chemical vs electrical) and far less precise ( iirc something comparable to 4 - 7x less precise than 32bit float) than physical neurons.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#64

Post by htrp » Fri, Oct 28, 2022, 1:06 PM UTC

bravura wrote: post #19

Animats wrote: post #6

This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…

Imagine if the immediate outcome of AI is not that we replace taxi drivers, dishwashers, and factory workers, but instead we displace most knowledge-worker white collar jobs, like quant and software engineer? There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve. Whereas…

Those "Complicated" tasks are all built in artificially constrained systems with limited degrees of variability, which is perfect for an algorithm to learn.

Those "Simple" tasks have so much variation in them that it takes a billion+ years of evolution + the genetic pretraining to be able to perform.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#65

Post by naasking » Fri, Oct 28, 2022, 1:07 PM UTC

jw1224 wrote: post #61

Galaxeblaffer wrote: post #58

Randomly watched this yesterday https://www.youtube.com/watch?v=hXgqik6HXc0&ab_channel=LexFr... where Roger Penrose argues that we're missing something fundamental about consciousness and his best bet is a structure called the microtubules. This talk reminded me of my own research into "AI" back in the 00's and that it's almost impossible to talk about AI since everybody has a different idea as to whay AI is, yes i k…

Roger Penrose & Stuart Hammeroff’s “Orchestrated Objective Reduction” theory [1] is fascinating and really captured my imagination when I came across it. But like almost all scientific theories tackling The Hard Problem, it’s built on the assumption that matter gives rise to consciousness. As time goes by and my own understanding deepens, I’m becoming more and more convinced that this assumption is wrong. Instead we…

> Idealism is still compatible with the material world, but it seems futile to search for “the experiencer” within the experience itself.

You're right, that's why the concept of "the experiencer" is ultimately an illusion. It's the same sort of illusion as "tables" and "a day job". None of these concepts fundamentally exist in physics, they are labels we apply to loosely defined categories of observations.

Ultimately, Descartes was wrong, "I think therefore I am" is false because it's circular; it assumes the existence of "I" to conclude that "I" exists. The fallacy-free version is "this is a thought therefore thoughts exist", and as you can see, no "I" can be inferred.

If you want to understand what sort of answer neuroscience is starting to provide to the hard problem, I recommend this paper:

A conceptual framework for consciousness, https://pnas.org/doi/10.1073/pnas.2116933119

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#66

Post by CuriouslyC » Fri, Oct 28, 2022, 1:12 PM UTC

rbanffy wrote: post #59

Remember growth is exponential - we won't recognize the next revolution because we'll still be dealing with the fallout of the previous one. Or previous dozen.

Incorrect. Any growth in a system of finite resources is sigmoidal, with an exponential portion early in the curve before diminishing returns kicks in.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#67

Post by CuriouslyC » Fri, Oct 28, 2022, 1:17 PM UTC

Jack000 wrote: post #57

sn41 wrote: post #22

As a theory person who usually explains O notation using concrete numbers, the degree of the neural network in our brain is approx 7000. Taking approx 86 billion ~ 100 billion, this itself is a graph with approx 6x10^(14) edges - does AGI proponents really hope to be able to do this? I am genuinely curious to know : is there some simplifying assumption which makes things faster?

You can make the same argument for cats, dogs and other mammals, which do have embodied intelligence but not the skills we typically associate with general intelligence (language, deductive reasoning, math, etc). Raw neuron count is only loosely associated with intelligence, which is highly variable even between individual humans with nearly equivalent neurons. Our brains are made of tiny little animals because that'…

Raw neuron count is only loosely associated with abstract problem solving intelligence. A cat's neural network is incredibly well optimized for things that cats care about.

Brains are very well optimized for computation/energy (while also being self replicating and self repairing), the tasks researchers care about just aren't the ones evolution cares about.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#68

Post by wrp » Fri, Oct 28, 2022, 1:18 PM UTC

Galaxeblaffer wrote: post #58

Randomly watched this yesterday https://www.youtube.com/watch?v=hXgqik6HXc0&ab_channel=LexFr... where Roger Penrose argues that we're missing something fundamental about consciousness and his best bet is a structure called the microtubules. This talk reminded me of my own research into "AI" back in the 00's and that it's almost impossible to talk about AI since everybody has a different idea as to whay AI is, yes i k…

Alwyn Scott's Stairway to the Mind (1995) has an accessible critique of Penrose's theory from the perspective of neurophysics. Basically, he argues that neuronal activity is on such a large time and energy scale that quantum effects are unlikely to be relevant.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#69

Post by naasking » Fri, Oct 28, 2022, 1:20 PM UTC

sn41 wrote: post #25

zamfi wrote: post #24

Earlier quoted context omitted.

That’s only 3 orders of magnitude off from today’s largest models like PaLM (5x10^11 parameters), a gap that’s narrowed by 3 orders of magnitude just since 2019. How far away do you think we are, exactly?

Thank you for this information. I did not know this. But my view (I may be wrong), is that AGI is too resource-intensive to be within the reach of normal computing of the ordinary user for at least 2 decades.

> is that AGI is too resource-intensive to be within the reach of normal computing of the ordinary user for at least 2 decades.

Hardware is still accelerating exponentially in density, albeit a bit slower. What you're not considering is that algorithmic improvements in machine learning are outpacing hardware improvements.

For instance, NVidia recently revealed how to switch from 32-bit floats to 16-bit floats with no perceptible loss in effectiveness, and they're working on 8-bit floats next. That's a full doubling in number of parameters in your model in only a single step. Other improvements are refinements to language models themselves to reduce overfitting and boost effectiveness with fewer parameters.

Arguably a machine learning model will achieve parity with human neuron density, in terms of number parameters, within the next decade. What that actually means is unclear.

Re: Artificial Intelligence: The Revolution Hasn’t Happened Yet (2018)

#70

Post by CuriouslyC » Fri, Oct 28, 2022, 1:21 PM UTC

goatlover wrote: post #15

cowtools wrote: post #11

Earlier quoted context omitted.

or perhaps just increased computing power

How does increasing computing power help with intuition/common sense about the world? Computing power isn't magic. It has to have a way to understand the world the way animals do.

Intuition and common sense are the result of latent learning via experience. A model with the right architecture could learn the same things given the training data.