Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let me clear a huge misunderstanding

twitter.com

Post reply on HN
Print view View original

71–80 of 109 posts

Re: Let me clear a huge misunderstanding

#71

Post by orwin »

hackerlight wrote: post #67
kgwgk wrote: post #35

Earlier quoted context omitted.

If the "couple holding hands and walking" one is the "Beautiful, snowy Tokyo city is bustling. ..." look at the traffic on the left side of the frame: https://www.youtube.com/watch?v=ezaMd4l_5kw We also have the spontaneous creation and annihilation of wolves and the shape-shifting chair: https://www.youtube.com/watch?v=jspYKxFY7Sc https://www.youtube.com/watch?v=lfbImB0_rKY

If we are talking analogies, this is just Sora forgetting because of limitations of how the network handles the autoregressive dynamics. When they make a bigger version of Sora this will happen less. Sora aleady has unprecedented object permanence, see the woman walking in Tokyo scene where signs and people are reconstructed after two seconds of occlusion. Soon we will have object permanence following ten or more sec…

Our brain can't work that in our long-term memory btw, that why each time we remember something, we change minor aspects of said thing.

Re: Let me clear a huge misunderstanding

#72

Post by ml-anon »

optimalsolver wrote: post #59
ml-anon wrote: post #49

LeCun is such a hack and is guilty exactly the same hype as OpenAI. Firstly his insistence on “self supervised learning” which is just a wrong and unhelpful rebranding of existing methodologies. Followed by talking about VicREG as if it’s a meaningful contribution and not just hacked together crap which is not only theoretically unfounded but plain nonsensical. Followed again by his “JEPA” work which again is just a…

>“self supervised learning” which is just a wrong and unhelpful rebranding of existing methodologies What are those methodologies?

In the 90’s and 2000’s we called it unsupervised learning.

In fact there is a direct formal equivalence between many so-called self supervised learning techniques and matrix factorization. And literally no one would claim that matrix factorisation is anything other than unsupervised.

Similarly Yann makes such a big deal about not doing contrastive learning when contrastive methods and his JEPA nonsense can also be shown to be formally equivalent. He’s a grifter like the rest of them. There’s a reason why he basically holds no power at Meta and hasn’t been in charge of AI research there in a long time.

Re: Let me clear a huge misunderstanding

#73

Post by Havoc »

>does not indicate that a system understands the physical world.

That's a bit like saying the chatbots aren't actually intelligent. Sure, but there is at least a plausible illusion of it & even that has usefulness.

The same applies to physical world. e.g. Look at the reflections in the SORA demo. They're not right, but they're also not entirely wrong either. That to me suggests usefulness in approximating the physical world

Re: Let me clear a huge misunderstanding

#74

Post by kgwgk »

orwin wrote: post #71
hackerlight wrote: post #67

Earlier quoted context omitted.

If we are talking analogies, this is just Sora forgetting because of limitations of how the network handles the autoregressive dynamics. When they make a bigger version of Sora this will happen less. Sora aleady has unprecedented object permanence, see the woman walking in Tokyo scene where signs and people are reconstructed after two seconds of occlusion. Soon we will have object permanence following ten or more sec…

Our brain can't work that in our long-term memory btw, that why each time we remember something, we change minor aspects of said thing.

It's also what happens when we dream: everything is fluid. Things appear and disappear, people and places become someone or somewhere else, reading is difficult and hands are distorted.

Re: Let me clear a huge misunderstanding

#75

Post by benreesman »

ml-anon wrote: post #49

LeCun is such a hack and is guilty exactly the same hype as OpenAI. Firstly his insistence on “self supervised learning” which is just a wrong and unhelpful rebranding of existing methodologies. Followed by talking about VicREG as if it’s a meaningful contribution and not just hacked together crap which is not only theoretically unfounded but plain nonsensical. Followed again by his “JEPA” work which again is just a…

He won the fucking Turing. I think you can rock your CV alongside “he’s such a hack”.

I’ve met the guy a few times (briefly), and I’m aware of his general vibe. I don’t agree with him about everything, but “hack” is absurd, and I’m not posting about who is a hack (or even a crook) under a burner alt.

What’s your Fields medal for?

Re: Let me clear a huge misunderstanding

#77

Post by throw310822 »

dkjaudyeqooe wrote: post #64
OJFord wrote: post #56

Earlier quoted context omitted.

I think that's my point? You were willing to say it doesn't apply without a definition?

I'm saying if you want to apply it outside human experience you need an concrete definition otherwise you can call anything you like 'understanding' which is what's happening.

If for you the definition of "understanding" is "something that only humans can do" then your statement about AI is totally pointless: of course AI doesn't "understand", but at the same time it might do something that is perfectly equivalent and that "only machines can do".

Re: Let me clear a huge misunderstanding

#78

Post by throw310822 »

TheOtherHobbes wrote: post #68
justanotherjoe wrote: post #51

Earlier quoted context omitted.

diffusion models can reliably draw a cat when prompted a cat and given a random noise. Sure it's deterministic, but it can work with any random noise in a explorative kind of way. I'd say it's very general in it's 'understanding' of a cat.

It has no "understanding" of a cat. It's an associative store with soft edges that pulls out compressed cat representations when given the noun "cat". The key store includes nouns, adverbs, verbs, and adjectives, and style abstractions, and there are mappings into the store that link all of those. But they're very limited, and if you prompt with a relationship that isn't defined you get best-guess, which will either…

> It's an associative store with soft edges that pulls out compressed cat representations when given the noun "cat".

And how do you know that this is not what "understanding" is? To me, understanding the concept of a cat is exactly to immediately recall (or have ready) all the associations, the possibilities, the consequences of the "cat" concept. If you can make up correct sentences about cats and conduct a reasonable conversation about cats, it means that you understand cats.

Re: Let me clear a huge misunderstanding

#79

Post by justanotherjoe »

TheOtherHobbes wrote: post #68
justanotherjoe wrote: post #51

Earlier quoted context omitted.

diffusion models can reliably draw a cat when prompted a cat and given a random noise. Sure it's deterministic, but it can work with any random noise in a explorative kind of way. I'd say it's very general in it's 'understanding' of a cat.

It has no "understanding" of a cat. It's an associative store with soft edges that pulls out compressed cat representations when given the noun "cat". The key store includes nouns, adverbs, verbs, and adjectives, and style abstractions, and there are mappings into the store that link all of those. But they're very limited, and if you prompt with a relationship that isn't defined you get best-guess, which will either…

I don't see what's so problematic with it. I doubt the model is actually confusing trees and branches with birds. It has associations, but humans do too. If I ask a human to draw a demon, the background would not be an office?

Also the complain about 'made of' not being in the training data. Humans who never saw a bird can not draw a bird. Why is that saying something about the model?

I'm not saying that diffusion models act like humans. And I was talking specifically about image generation. My usage of the word understanding is in the task of image generation. I'm not even talking about 'made of', or 'birds'. Just 'cats' and 'hats'. If it can understand 1 thing, it can understand others, but they are not always in the training data.

This is all a non-problem. It kinda remind me of the discussion of what constitutes a 'male', or 'female'. All i want is to refer this one property that i observe in diffusion models. Which is what language is, reference. If you are so covetous of the word 'understand', then provide an alternative to refer to this property and i will gladly use it.

https://imgur.com/a/fuS8kcf

Re: Let me clear a huge misunderstanding

#80

Post by bamboozled »

dkjaudyeqooe wrote: post #43

OpenAI willfully fuels hype of all sorts and lets people extrapolate without basis or limit, because it's hugely profitable for them. LeCun is the voice of reason trying to point out the limitations of the current technology and fundamental questions that need to be answered. Until there is something more than cute demos, like an actual path forward that can implement what people are hyping, or better still working e…

LeCun is the voice of reason trying to point out the limitations of the current technology and fundamental questions that need to be answered.

I think this is the more likely true, but less popular take as well.

Post reply on HN