Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let me clear a huge misunderstanding

twitter.com

Post reply on HN
Print view View original

101–109 of 109 posts

Re: Let me clear a huge misunderstanding

#101

Post by throw310822 »

Jensson wrote: post #94
throw310822 wrote: post #78

Earlier quoted context omitted.

> It's an associative store with soft edges that pulls out compressed cat representations when given the noun "cat". And how do you know that this is not what "understanding" is? To me, understanding the concept of a cat is exactly to immediately recall (or have ready) all the associations, the possibilities, the consequences of the "cat" concept. If you can make up correct sentences about cats and conduct a reasonab…

> If you can make up correct sentences about cats and conduct a reasonable conversation about cats, it means that you understand cats. No, plenty of humans can have reasonable conversations about things with zero understanding about them. We know they don't understand because when put in a situation in practice they fail to use the things they talked about. Understanding means you can apply what you know, not only ta…

It's interesting, can you give me some examples of such cases?

Re: Let me clear a huge misunderstanding

#102

Post by benreesman »

ml-anon wrote: post #85
benreesman wrote: post #75

Earlier quoted context omitted.

He won the fucking Turing. I think you can rock your CV alongside “he’s such a hack”. I’ve met the guy a few times (briefly), and I’m aware of his general vibe. I don’t agree with him about everything, but “hack” is absurd, and I’m not posting about who is a hack (or even a crook) under a burner alt. What’s your Fields medal for?

Just because he made a good thing decades ago doesn’t mean he has any clue about what is going on now. Many of us actually work in the field and are tired of the mindless hype and self promotion. LeCun is just upset people aren’t hyping his crap rather than OpenAIs. But I’m sure you can give a good reason why he calls it self supervised instead of unsupervised and JEPA instead of BYOL or data2vec and what his actual…

I didn’t say you can’t criticize household names with the highest honor in their field.

I’m saying that it takes some cheek to say there’s nothing original about results that many experts (who are nothing to do with me, and me in a narrow way) think are pretty hot shit.

I try (and sometimes fail) to confine my strident claims requiring strident evidence to topics I’m willing and able to get into the nitty gritty on, and bring my CV and reputation to the party as a courtesy. Anyone can post biased-sounding shit and hand-wave over the substance of the claim under an 11-month burner named with malice aforethought to talk shit and be able to disavow it.

I gave you a rather firm tap on the shoulder about how borderline at best this move is, and under the assumption you’ve got an HN main, you know this.

I had hoped you might put the effort that went into doubling down into explaining the apparently obvious prior art, the whataboutism kick flip into the Shockley Race IQ trope made zero people more enlightened and steered a contentious thread even further off the rails. Nobel laureates do in fact go off the friggin rails from time to time, but I don’t think LeCun is an interesting comparison to Kary Mulis past “AM Radio yelling”. I’m calling that one out.

Maybe elaborate on a pretty debatable claim instead?

Re: Let me clear a huge misunderstanding

#103

Post by benreesman »

radarsat1 wrote: post #19

The main thing I'm confused about by these comments is that as far as I understand, the Sora model (and many others like it) performs the diffusion process in latent space, and then translates this to pixels. So it's strange to me to claim that it doesn't have an abstract representation. But, maybe the latent space of a diffusion-VQVAE pipeline is fundamentally different from that of JEPA, I haven't read the relevant…

The claim, with admittedly limited consensus, is that a difference in degree becomes a difference in kind: we’ve known “forever in AI years” that restricting the inputs, and loss, and parameter dimensionality is right up there with the underlying architecture in terms of the properties those latent geometries exhibit. It’s even kind of a meme: the fingers problem (which neither Sora nor V-JEPA partisans seem to be claiming is “solved”).

I’m a little oversubscribed at the moment so I haven’t downloaded the weights and played around with V-JEPA. The fact I’m pointing out that I could should make it pretty clear which way I lean on this: everyone wants to make money, let’s be real, but some seem resigned to “infinite, government-enforced monopoly via OpenPhilanthropy bribery” isn’t going to fly, so let’s at least prove how the sausage is made.

It’s fairly uncontroversial that things in the neighborhood of a “bottlenecked” VAE are often forced to exploit structure if it’s there.

This (claimed) result is about a way to exploit more of this structure/symmetry economy (greedy-ish optimizer) to pull the latent representations into yet a higher “effective regime” than yet demonstrated, with excellent properties around machine economics.

Representation learning isn’t new (though LeCun is an acknowledged pioneer in it), and constraints as a powerful tool isn’t new (causal making in attention architectures shouldn't ruffle many feathers).

Self/semi/unsupervised learning isn’t novel. But likewise, not are they k-equivalent synonyms: Dean et al. were distinguishing between Continuous Bag of Words and skip-gram in word2vec in 2013.

But it does cool shit (I like the I-JEPA reconstructions as a go to slide deck raster), and it’s available weight.

I think the truth shall soon emerge.

Re: Let me clear a huge misunderstanding

#104

Post by ml-anon »

benreesman wrote: post #102
ml-anon wrote: post #85

Earlier quoted context omitted.

Just because he made a good thing decades ago doesn’t mean he has any clue about what is going on now. Many of us actually work in the field and are tired of the mindless hype and self promotion. LeCun is just upset people aren’t hyping his crap rather than OpenAIs. But I’m sure you can give a good reason why he calls it self supervised instead of unsupervised and JEPA instead of BYOL or data2vec and what his actual…

I didn’t say you can’t criticize household names with the highest honor in their field. I’m saying that it takes some cheek to say there’s nothing original about results that many experts (who are nothing to do with me, and me in a narrow way) think are pretty hot shit. I try (and sometimes fail) to confine my strident claims requiring strident evidence to topics I’m willing and able to get into the nitty gritty on,…

This is the most hilarious internet tough guy shit I've ever seen.

Like literally just read the papers I mentioned. Or don't, I don't care. I have literally no clue who you are so maybe take your creepy-as-fuck "firm tap on the shoulder" and wanting to see my "cv" shit somewhere else.

Re: Let me clear a huge misunderstanding

#105

Post by benreesman »

ml-anon wrote: post #104
benreesman wrote: post #102

Earlier quoted context omitted.

I didn’t say you can’t criticize household names with the highest honor in their field. I’m saying that it takes some cheek to say there’s nothing original about results that many experts (who are nothing to do with me, and me in a narrow way) think are pretty hot shit. I try (and sometimes fail) to confine my strident claims requiring strident evidence to topics I’m willing and able to get into the nitty gritty on,…

This is the most hilarious internet tough guy shit I've ever seen. Like literally just read the papers I mentioned. Or don't, I don't care. I have literally no clue who you are so maybe take your creepy-as-fuck "firm tap on the shoulder" and wanting to see my "cv" shit somewhere else.

We all go out of bounds on HN norms from time to time, and myself more than many.

I generally appreciate the feedback that I’ve landed there.

This isn’t seeming to be going anywhere useful and I’m going to respectfully bow out. If you’ve got to have the last word, go to town: but do it by posting the links to the papers for those who haven’t read them.

I’ve pulled some dumb stunts on HN, even some recently, but I’ll stand behind there being no Navy Seal Copypasta flex here for the record.

Be well ml-anon.

Re: Let me clear a huge misunderstanding

#106

Post by namaria »

throw310822 wrote: post #91
inference-lord wrote: post #90

Earlier quoted context omitted.

I guess you understand wolves, do you think this video demonstrates understanding of how wolves work? https://www.youtube.com/watch?v=jspYKxFY7Sc

Certainly. It demonstrates it knows how they look, how they move, what type of behaviour they show at a certain age, in what kind of environment you might see them. What it doesn't seem to have is enough persistence to remember how many there are. But the idea of an animate, acting wolf is there, no doubt about it.

Models can't understand anything but the dataset they are trained on. This is very obvious from the plausible looking wolf cubs moving plausibly while behaving in absurd ways when it comes to real life wolves, or real life anythings for that matter. Models are compressing huge planet scale datasets and spitting out plausible instances of bytes that could belong to the training dataset, but very obviously fail to grasp any real world understanding of what is represented by those bytes.

Re: Let me clear a huge misunderstanding

#107

Post by namaria »

throw310822 wrote: post #101
Jensson wrote: post #94

Earlier quoted context omitted.

> If you can make up correct sentences about cats and conduct a reasonable conversation about cats, it means that you understand cats. No, plenty of humans can have reasonable conversations about things with zero understanding about them. We know they don't understand because when put in a situation in practice they fail to use the things they talked about. Understanding means you can apply what you know, not only ta…

It's interesting, can you give me some examples of such cases?

If you don't believe that is true, how can you explain programmer recruitment? If discussing something cogently is showing real understanding, any 10 minute discussion would be enough to make a hire decision.

Re: Let me clear a huge misunderstanding

#108

Post by namaria »

kgwgk wrote: post #74
orwin wrote: post #71

Earlier quoted context omitted.

Our brain can't work that in our long-term memory btw, that why each time we remember something, we change minor aspects of said thing.

It's also what happens when we dream: everything is fluid. Things appear and disappear, people and places become someone or somewhere else, reading is difficult and hands are distorted.

Because these systems are dreaming about their datasets. Or hallucinating about it, as people have decided to call lately. I won't say this is a dead end. I will say we are very, very short of any sort of actual intelligence.

Re: Let me clear a huge misunderstanding

#109

Post by Doxin »

Jensson wrote: post #88
bonzaidrinkingb wrote: post #29

Earlier quoted context omitted.

You've seen the demos of a couple holding hands and walking, or the museum shots where all the paintings maintain coherence, or a woman temporarily obscuring a street sign. Or you haven't seen those demos. Either way...

In the couple holding hands videos you see the people walking in front duck into a wall and disappear, the girl in front walks into the fence and disappears, another girl walks straight through that fence. These aren't just issues with forgetting, it completely doesn't understand how things works. It draws a fence but then doesn't understand that people can't walk through it. Or the video with the dog, that dog phase…

The primary thing I noticed is that it doesn't quite seem to grasp that time runs forward either. In the video with the dogs in the snow you can see snow getting kicked up in reverse. I.E. snow arcs through the air and lands right as a paw gets placed.

Kind of made me wonder how these videos would look run backwards, but not enough to figure out how to make them run backwards.

EDIT: wow, the "backwards" physics is especially noticeable in the chair video[0]. Aside from the chair morphing wildly, notice how it floats and bounces around semi-physically. Clearly some issues grasping cause and effect.

[0] https://www.youtube.com/watch?v=lfbImB0_rKY

Post reply on HN