Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google DeepMind CEO says some form of AGI possible in a few years

wsj.com

Post reply on HN
Print view View original

221–230 of 388 posts

Re: Google DeepMind CEO says some form of AGI possible in a few years

#221

Post by VHRanger »

somewhereoutth wrote: post #164
earthnail wrote: post #123

Earlier quoted context omitted.

I disagree with you. > Essentially all we've done is pushed the basic model proposed by linear regression to it's absolutely limits No, we haven't pushed linear regression to its limits. If it was only linear regression, it wouldn't work. Neural networks need a non-linearity to model complex things. The beauty is that given an infinite series of nonlinearities, one can model any mathematical function. In practice we…

You don't need non-linearities, an infinite series of sine functions is enough to model any function. For extraordinary claims ('intelligence'), the burden of proof is on those making the claim, not on others to prove the negative.

You can even do it in a linear regression. Just add enough polynomial terms (x^a) and interaction terms (x1 * x2). The end model looks something like:

y = b + x1 + x1^2 + x1^3 + ... + x1 * x2 + (x1 * x2)^2 + ... + x2 + x2^2 + ...

By that point you're making a Taylor approximation of the latent function through linear space, which is also a universal approximator.

So the commenter above is wrong -- neural networks are indeed just glorified linear regression from this point of view.

The main difference is that this kitchen sink regression is computationally inefficient which neural nets are extremely efficient computationally.*

Re: Google DeepMind CEO says some form of AGI possible in a few years

#222

Post by sebastiennight »

vidarh wrote: post #201
riffraff wrote: post #166

Earlier quoted context omitted.

I think this is the culprit: ChatGPT does not understand , as proven by hallucinations, by being wrong on certain questions or puzzle etc.. It just seems to understand. This is useful, and deeply impressive, but it's not the same thing.

Humans are confidently wrong about all kinds of things all the time. We don't call it hallucination other than for very specific, limited subsets. We call each other names over it, call it mistakes, stupidity, or lies. In fact, we structure large parts of society around it (the presence of multiple contradictory world religions means the majority of the worlds population goes through life being confidently wrong). It…

I think that the ChatGPT model (at least chatgpt-3.5-turbo) has gotten impressively better at this.

- In my most recent tests, it will tell you when the data you've provided doesn't match the task (instead of inventing an answer out of thin air).

- It will also add (unprompted) comments/notes before or after the result to disclaim a plausible reason why certain choices have been made or the answer isn't complete.

You have to take into account that not everyone wants the model to not hallucinate. There is a lot of competing pressure:

- Some people would like the model to say "As an AI model trained by OpenAI, I am not qualified to provide an answer because this data is not part of my training set" or something similar, because they want it to only talk when it's sure of the data. (I personally think this use case - using LLMs as search engines/databases of truth - is deeply flawed and not what LLMs are for ; but for a large enough GPT-n, it would work perfectly fine. There is a model size where the model would indeed contain the entire uncompressed Internet, after all)

- Some people want the model to never give such a denial, and always provide an answer in the require format, even if it requires the model to "bullshit" or improvise a bit. An example is, if that as a business user I provide the model with a blog article and ask for metadata in a JSON structure, I want the model to NEVER return "As an AI model..." and ALWAYS return valid JSON, even if the metadata is somewhat shaky or faulty. Most apps are more tolerant to BS than they are to empty/invalid responses. That's the whole reason behind all of those "don't reply out of character/say you are an AI" prompts you see floating around (which, in my experience, are completely useless and do not affect the result one bit)

So the reinforcement is constantly going in those opposite directions.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#224

Post by HKH2 »

FredPret wrote: post #117
russelldjimmy wrote: post #103

Earlier quoted context omitted.

Off topic - a pet peeve of mine is seeing humans termed as “irrational”. Please forgive my rant, as it is not personally targeted at you. We only seem “irrational” when we are talking about a narrow view of “rationality”, i.e., as defined by the cold hard logic of machines. We do not question why we have this definition of rationality. Our “irrationality” simply seems so because we have not bothered to understand the…

Absolutely on point. I would argue that irrationality isn't even possible. If person A thinks or behaves "irrationally" according to person B, there is simply a difference in perception between the two. A large percentage of those perceptions are created inside one's own mind which may or may not be aligned with the rest of the universe.

What about cognitive dissonance?

Re: Google DeepMind CEO says some form of AGI possible in a few years

#225

Post by grumple »

charcircuit wrote: post #127
grumple wrote: post #65

Earlier quoted context omitted.

Aren't the best self driving cars causing traffic jams in SF because they can't handle everyday tasks? And can any self driving car handle suboptimal environments like snow? Or just unmarked roads? Or... left turns in traffic?

Just because you can't handle every situation that could occur when driving it doesn't mean you don't have the ability to drive. I can't parallel park, but I would still consider myself able to drive. I make mistakes driving where I'm not supposed to yet I'm still a driver.

But you're not stopping in the middle of the street for long periods in confusion because of a common situation. There's levels to this and the self driving cars are not yet at an acceptable level.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#226

Post by Lerc »

wesammikhail wrote: post #140
earthnail wrote: post #123

Earlier quoted context omitted.

I disagree with you. > Essentially all we've done is pushed the basic model proposed by linear regression to it's absolutely limits No, we haven't pushed linear regression to its limits. If it was only linear regression, it wouldn't work. Neural networks need a non-linearity to model complex things. The beauty is that given an infinite series of nonlinearities, one can model any mathematical function. In practice we…

Andrej Karpathy, the director of AI at tesla, published an AMAZING video about how the current models are built which I think everyone in the tech space should watch. It is intuitive, easy to follow and quite frankly the best video I have seen on the topic. Here is the link: https://www.youtube.com/watch?v=kCc8FmEb1nY&list=FL2tbfd7UpJ... If you conclude, after watching this, that this is all intelligence is - namely…

I'm ok with believing that all it takes to make intelligence (probably not the only way) is a sufficiently large neural net with the right architecture and weights.

I think it is easy to distracted by the specific mechanisms by which these models work, but most of the technological detail is because we want something to happen on systems at the scale of what we can actually build. We simply can't build a human brain scale neural net yet. We build what we can and besides maybe with all this research we will figure something significant about what intelligence actually is.

The notion "This can't be all thought is" is as old as the idea of AI. I think it informed Turing when he proposed the Imitation Game. The insight is that people would be resistant to the idea of a bunch of simple things stuck together becoming thinking until they were faced with something that behaves sufficiently indistinguishable from themselves that to doubt that they were thinking would be akin to doubting everyone you meet.

In the end some people won't even accept an AI that does everything a human does as actually thinking, but then again some people are actually solipsistic.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#227

Post by novaRom »

krisoft wrote: post #179
PheonixPharts wrote: post #41

As someone who has worked in the field of AI/ML for quite awhile now, the problem with current AGI predictions is ML hasn't done anything new since the 80s (or arguably earlier). At the end of the day all ML is using gradient descent to do some sort of non-linear projection of the data on to a latent space, then doing some relatively simple math in this latent space to perform some task. Personally I think the limits…

> ML hasn't done anything new since the 80s Lol, let me ask ChatGPT what it thinks about that. :) > I don't believe we've seen any evidence of this What kind of evidence would you like to have? Do you want a mathematical proof or what? There is evidence that we are making forward progress in solving problems which were previously in the domain of human cognition. There is evidence that yesterday's "impossible" proble…

> ML hasn't done anything new since the 80s

Schmidhuber, is it you?

But seriously, it's partially true most techniques we use today could be found initially envisaged in 90th ANN-related papers. I think Geoffrey Hinton summarized most of them well in his famous "Neural Networks for Machine Learning" lectures. Essentially, what is available today is compute resources unimaginable in 90th, so scaling up from a shallow 2-layered Multi Layer Perceptron to something like 96-layered deep architecture is possible only recently. We also found some of the tricks work better than others in practice when we scale up (like *ELU non-linearity, layer-norm, residual connections). What stays the same however is the general approach: training and validation sets, cross entropy loss, softmax, learnable parameters based on data-in/data-out training pairs, and differentiation chain rule. IMO this requires some innovative revision, especially generalization is still very weak in all architectures today.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#228

Post by 300bps »

deagle50 wrote: post #75
valine wrote: post #39

Earlier quoted context omitted.

Or, more optimistically, AGI will accelerate scientific progress and we will have fusion, quantum computers, and self driving cars shortly after. If AIs had the tendency to recursively devour worlds you would expect at least one alien AI to have conquered our galaxy by now. But the sky is quiet, so either we are the very first species to get this far, or there’s a natural ceiling to the pace of technological progress…

The directives that mandated eliminating its creators don't necessarily lead to interstellar conquest. Or more likely IMO, we're just too far apart and interstellar travel isn't feasible even for artificial super intelligence.

Here’s a paper that shows how a civilization could colonize the galaxy in under 300 million years:

https://arxiv.org/pdf/1902.04450.pdf

Here’s a YouTube visualization of the timeline:

https://youtu.be/hNMgtRf0GOg

Re: Google DeepMind CEO says some form of AGI possible in a few years

#229

Post by HKH2 »

circuit10 wrote: post #112
HKH2 wrote: post #82

Earlier quoted context omitted.

Despite our best efforts, we are deeply irrational. Our thinking is based on instinct, not on core principles; it's a top-down approach driven by feelings.

Why is instinct not computable? That seems way easier to compute than rational thinking based on principles, it's just "if this, do that" and machine learning should be able to do that easily

If you could isolate processes like that, sure.

Re: Google DeepMind CEO says some form of AGI possible in a few years

#230

Post by kumarvvr »

vidarh wrote: post #202
kumarvvr wrote: post #156

Earlier quoted context omitted.

Is it understanding or, generating closest possible set of tokens that are accepted as a response to an received input of tokens?

How do we know those two are different?

Understanding and assimilation can lead to generating relations between disjoint sets of tokens.

For example, "squeeze a tube to cut water flow" and "put pressure on a deep wound to stop blood loss" can only be related, if not already in the training data, if there is understanding and intelligence.

The ability to do that is intelligence.

Otherwise, it's just a search and optimization problem.

Post reply on HN