Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let me clear a huge misunderstanding

twitter.com

Post reply on HN
Print view View original

61–70 of 109 posts

Re: Let me clear a huge misunderstanding

#61

Post by loveparade »

optimalsolver wrote: post #59
ml-anon wrote: post #49

LeCun is such a hack and is guilty exactly the same hype as OpenAI. Firstly his insistence on “self supervised learning” which is just a wrong and unhelpful rebranding of existing methodologies. Followed by talking about VicREG as if it’s a meaningful contribution and not just hacked together crap which is not only theoretically unfounded but plain nonsensical. Followed again by his “JEPA” work which again is just a…

>“self supervised learning” which is just a wrong and unhelpful rebranding of existing methodologies What are those methodologies?

Self-supervised learning is such a trivial concept that it doesn't need a new term. It's just marketing/PR speech to attract attention. You could just call it "generating labels from data"

All these stupid new terms of trivial concepts confuse people who are new to the field. That's nothing new though, research has always been like this, people love to make up new phrases that they can own and sell as novel ideas to reviewers, even if they're just trivial renamings. It's nothing more than a PR game.

Re: Let me clear a huge misunderstanding

#62

Post by School-Cotton »

awaythrow999 wrote: post #45
stavros wrote: post #44

Earlier quoted context omitted.

Is it really impossible? I clicked the link and saw the content fine. If you mean "impossible without signing up", I'd say it's within your rights to not want to sign up, but it's a very uncharitable definition of "impossible".

Impossible for me yes. Am getting prompted by a sign-up screen to join twitter.

Why is it impossible for you to sign up for Twitter?

Re: Let me clear a huge misunderstanding

#63

Post by hackerlight »

justanotherjoe wrote: post #16
vasco wrote: post #14

Earlier quoted context omitted.

I'm nowhere near an expert but it seemed like he was claiming true understanding of latent space is needed for generating coherent continuations, but Sora demo already has longish videos that are coherent. It's hard for me not to think this is someone just trying to still be right when they are wrong, but I may misunderstand.

The way i understand it is that Sora is mostly just 'moving pictures' with no rhyme or reason. Yann Lecun is interested in videos that tell a 'story', with cause and effect. Like a magician putting his hand into a top hat and pulling out a rabbit, kind of video.

I want him to answer how Sora can do water simulations without having a model of part of physics.

How can Sora predict where the waves and ripples should go? Is it just "correlations not causal", whatever that means.

Re: Let me clear a huge misunderstanding

#64

Post by dkjaudyeqooe »

OJFord wrote: post #56
dkjaudyeqooe wrote: post #55

Earlier quoted context omitted.

Understanding, in this context, is what humans do, by definition. Until you have a concrete definition of what understanding is you can't apply it to anything else. Informal definitions of understanding by those who experience it aren't very useful at all.

I think that's my point? You were willing to say it doesn't apply without a definition?

I'm saying if you want to apply it outside human experience you need an concrete definition otherwise you can call anything you like 'understanding' which is what's happening.

Re: Let me clear a huge misunderstanding

#65

Post by optimalsolver »

loveparade wrote: post #61
optimalsolver wrote: post #59

Earlier quoted context omitted.

>“self supervised learning” which is just a wrong and unhelpful rebranding of existing methodologies What are those methodologies?

Self-supervised learning is such a trivial concept that it doesn't need a new term. It's just marketing/PR speech to attract attention. You could just call it "generating labels from data" All these stupid new terms of trivial concepts confuse people who are new to the field. That's nothing new though, research has always been like this, people love to make up new phrases that they can own and sell as novel ideas to…

If people didn't consider it a useful pointer to a certain set of ideas, it wouldn't have caught on in the ML community.

That's how language works.

Re: Let me clear a huge misunderstanding

#66

Post by gtirloni »

hackerlight wrote: post #63
justanotherjoe wrote: post #16

Earlier quoted context omitted.

The way i understand it is that Sora is mostly just 'moving pictures' with no rhyme or reason. Yann Lecun is interested in videos that tell a 'story', with cause and effect. Like a magician putting his hand into a top hat and pulling out a rabbit, kind of video.

I want him to answer how Sora can do water simulations without having a model of part of physics. How can Sora predict where the waves and ripples should go? Is it just "correlations not causal", whatever that means.

Generative AI is already full of misunderstandings. From people claiming it "understands" to now that it "simulates".

I'm no math wiz and my training in statistics is severely lacking but it feels like people need to review what they think it's possible with Generative AI because we are so far from understanding and AGI that my head hurts every time these words show up in a discussion.

Re: Let me clear a huge misunderstanding

#67

Post by hackerlight »

kgwgk wrote: post #35
bonzaidrinkingb wrote: post #29

Earlier quoted context omitted.

You've seen the demos of a couple holding hands and walking, or the museum shots where all the paintings maintain coherence, or a woman temporarily obscuring a street sign. Or you haven't seen those demos. Either way...

If the "couple holding hands and walking" one is the "Beautiful, snowy Tokyo city is bustling. ..." look at the traffic on the left side of the frame: https://www.youtube.com/watch?v=ezaMd4l_5kw We also have the spontaneous creation and annihilation of wolves and the shape-shifting chair: https://www.youtube.com/watch?v=jspYKxFY7Sc https://www.youtube.com/watch?v=lfbImB0_rKY

If we are talking analogies, this is just Sora forgetting because of limitations of how the network handles the autoregressive dynamics. When they make a bigger version of Sora this will happen less. Sora aleady has unprecedented object permanence, see the woman walking in Tokyo scene where signs and people are reconstructed after two seconds of occlusion. Soon we will have object permanence following ten or more seconds of occlusion. Then a minute. Then three minutes. Then we will figure out a trick to store long term memory. What will people say then?

Re: Let me clear a huge misunderstanding

#68

Post by TheOtherHobbes »

justanotherjoe wrote: post #51
dkjaudyeqooe wrote: post #40

Earlier quoted context omitted.

It doesn't understand a cat at all. Humans understand, models are deterministic functions with some randomness added in. Just because it appears to understand doesn't make it so.

diffusion models can reliably draw a cat when prompted a cat and given a random noise. Sure it's deterministic, but it can work with any random noise in a explorative kind of way. I'd say it's very general in it's 'understanding' of a cat.

It has no "understanding" of a cat. It's an associative store with soft edges that pulls out compressed cat representations when given the noun "cat". The key store includes nouns, adverbs, verbs, and adjectives, and style abstractions, and there are mappings into the store that link all of those.

But they're very limited, and if you prompt with a relationship that isn't defined you get best-guess, which will either be quite distant or contaminated with other values.

If you ask Dall-E for "a woman made of birds" you get a composite that also includes trees and/or leaves. Dall-E has values for "made of" and "birds" but its value representation for "birds" is contaminated with contextual trees and branches.

Leonardo doesn't have a value for "made of", so you get a woman surrounded by bird-like blobs.

To understand a cat in a human sense the store would have to include the shape, the movement dynamics in all possible situations, the textures, and a set of defining behaviours that is as complete as possible. It would also have to be able to provide and remember an object-constant instantiation of a specific cat that is clean of contamination.

SORA is maybe 10% of the way there. One of the examples doing the rounds shows some puppies playing in snow. It looks impressive until you realise the puppies are zombies. They have none of the expressions or emotions of a real puppy.

None of this is impossible, but training time, storage, and power consumption all explode the more information you try to include.

Re: Let me clear a huge misunderstanding

#69

Post by cinntaile »

awaythrow999 wrote: post #34

It's impossible to access this content (even more so now that nitter is dead). Should HN continue driving traffic to a site that is entirely inaccessible?

I know @dang has commented on the subject before but now that Nitter is dead, it should be considered hardwalled and Twitter links should be flagged imo. There is no real workaround anymore that doesn't involve signing up.

Re: Let me clear a huge misunderstanding

#70

Post by hackerlight »

gtirloni wrote: post #66
hackerlight wrote: post #63

Earlier quoted context omitted.

I want him to answer how Sora can do water simulations without having a model of part of physics. How can Sora predict where the waves and ripples should go? Is it just "correlations not causal", whatever that means.

Generative AI is already full of misunderstandings. From people claiming it "understands" to now that it "simulates". I'm no math wiz and my training in statistics is severely lacking but it feels like people need to review what they think it's possible with Generative AI because we are so far from understanding and AGI that my head hurts every time these words show up in a discussion.

Geoffrey Hinton and Yoshua Bengio says it can understand and we are close to AGI. Maybe you can explain why they're wrong instead of just saying it.
Post reply on HN