Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I disagree with Geoff Hinton regarding "glorified autocomplete"

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

11–20 of 279 posts

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#11

Post by hopfenspergerj »

overactor wrote: post #8

I think this misses the point made by Hinton. The point is that, if a model is to reach a certain level of autocomplete, it needs to understand what it is talking about. Imagine the result of what required deep thought from you is included in the training data, along with billions of other such exchanges. At some point the only way to improve training score is to develop the ability to "understand" things. (That is,…

I agree. We should prompt the model with the statement of the Riemann hypothesis. If the autocomplete is good, the model will output a proof.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#12

Post by satuke »

mbrumlow wrote: post #5

> If you want to be really good, you have to understand what’s being said. That’s the only way. This is simply not true. Predicting the next letter or word, or id you abstract it away from things that mean something to you, like the next color of a block in a long chain of colored blocks. You would realize that all we are doing is using statistics to predict what the next item might be. There simply is no need or req…

Isn't that exactly how humans learn to respond to stimuli? Don't we just try to predict the best next response to everything? Yes, It's statistics but the fun part is nobody is writing this statistical function by hand.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#13

Post by zeroonetwothree »

I found this post to be overly long and unclear in its focus. Also who is this person that only serves as an autocomplete in meetings? Maybe don’t go to those meetings then? In my meetings I’m mainly having to decide what our approach will be in some ambiguous problem space.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#14

Post by satuke »

coffeebeqn wrote: post #7
chpatrick wrote: post #4

Earlier quoted context omitted.

Why don't you ask it not to be?

The default style really annoys me with its verbosity and overuse of numbered lists. Describing a tone and length of responses and anything else you want - think step by step, first principals, show don’t tell - all work pretty well. You can even ask it to only reply with a Boolean true or false - which certainly doesn’t play to its strengths

Actually, that's a by-product of RLHF. A base model is usually not that verbose.

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#15

Post by patcon »

Yeah, I agree there are two types:

1) repeating things he's reflected in or seen in the past (more like autocomplete), or

2) mixing two semantic concepts that he's seen in the past, and using it as a bridge to walk over to get to the next word/concept (which is back to autocomplete mode)

The second is like crafting in Minecraft, instead of just taking things out of inventory (but all of that is happening in high dimensional semantic space :) )

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#16

Post by raincom »

zeroonetwothree wrote: post #9

We have quantum mechanics, a model of the universe that makes predictions with absurd levels of accuracy. Yet we don’t really understand “why” it works—-I don’t think we can claim that we really “understand” the universe. This disproves the claim that making accurate predictions implies understanding. Or perhaps a better way to put it is that the model has some understanding but a model doesn’t necessarily reflect un…

Best examples come from medicine. There are many drugs that work, but we don't understand how they work. For example: https://en.wikipedia.org/wiki/Category:Drugs_with_unknown_me...

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#17

Post by overactor »

hopfenspergerj wrote: post #11
overactor wrote: post #8

I think this misses the point made by Hinton. The point is that, if a model is to reach a certain level of autocomplete, it needs to understand what it is talking about. Imagine the result of what required deep thought from you is included in the training data, along with billions of other such exchanges. At some point the only way to improve training score is to develop the ability to "understand" things. (That is,…

I agree. We should prompt the model with the statement of the Riemann hypothesis. If the autocomplete is good, the model will output a proof.

No human can solve the Riemann Hypothesis. Why do you expect an AI to do it before you consider it able to understand things?

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#18

Post by [deleted] »

zeroonetwothree wrote: post #9

We have quantum mechanics, a model of the universe that makes predictions with absurd levels of accuracy. Yet we don’t really understand “why” it works—-I don’t think we can claim that we really “understand” the universe. This disproves the claim that making accurate predictions implies understanding. Or perhaps a better way to put it is that the model has some understanding but a model doesn’t necessarily reflect un…

[deleted]

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#19

Post by [deleted] »

zeroonetwothree wrote: post #13

I found this post to be overly long and unclear in its focus. Also who is this person that only serves as an autocomplete in meetings? Maybe don’t go to those meetings then? In my meetings I’m mainly having to decide what our approach will be in some ambiguous problem space.

[deleted]

Re: I disagree with Geoff Hinton regarding "glorified autocomplete"

#20

Post by marginalia_nu »

zeroonetwothree wrote: post #9

We have quantum mechanics, a model of the universe that makes predictions with absurd levels of accuracy. Yet we don’t really understand “why” it works—-I don’t think we can claim that we really “understand” the universe. This disproves the claim that making accurate predictions implies understanding. Or perhaps a better way to put it is that the model has some understanding but a model doesn’t necessarily reflect un…

What are you even asking when you're asking why it works. Ultimately, it works the way it does because that's the way things are. I'm not sure what sort of answers you expect when asking why physics acts in a particular fashion.

Also here's Feynman ranting about why questions: https://www.youtube.com/watch?v=36GT2zI8lVA

Post reply on HN