Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We accidentally solved robotics by watching 1M hours of YouTube

ksagar.bearblog.dev

Post reply on HN
Print view View original

101–110 of 183 posts

Re: We accidentally solved robotics by watching 1M hours of YouTube

#101

Post by shiroiuma »

shadowgovt wrote: post #72
marcus_holmes wrote: post #66

Earlier quoted context omitted.

There's an English phrase "hounded to death", meaning that someone was pursued and hassled until they died. It doesn't specify the cause of death, but I think the assumption would be suicide, since you can't actually die of fatigue. I think that's what was done to Aaron Swartz.

Many people have dealt with the law, with copyright infringement, even with gross amounts of it, and had the book thrown at them, and survived the experience. Swartz was ill. It is a tragedy he did not survive the experience, and indeed, trial is very stressful. But he was no more hounded than any defendant who comes under federal scrutiny and has to defend themselves in a court of law via the trial system. Kevin Mit…

Maybe someone should throw you in prison for a year on some BS made-up charges to see how well you survive it. We can use it as a data point for your argument.

Re: We accidentally solved robotics by watching 1M hours of YouTube

#102

Post by dchftcs »

rocqua wrote: post #95
dchftcs wrote: post #78

Pure vision will never be enough because it does not contain information about the physical feedback like pressure and touch, or the strength required to perform a task. For example, so that you don't crush a human when doing massage (but still need to press hard), or apply the right amount of force (and finesse?) to skin a fish fillet without cutting the skin itself. Practically in the near term, it's hard to sample…

On humans, you can generally see the force they apply by looking at strain.

The error margins will be huge, and for small enough force (like the skinning part or handling fine mechanical stuff) there's basically almost zero signal.

Re: We accidentally solved robotics by watching 1M hours of YouTube

#104

Post by a_t48 »

dimatura wrote: post #74

"why didn't we think of this sooner?", asks the article. Not sure who the "we" is supposed to be, but the robotics community has definitely thought of this before. https://robo-affordances.github.io/ from 2023 is one pretty relevant example that comes to mind, but I have recollections of similar ideas going back to at least 2016 or so (many of which are cited in the V-JEPA2 paper). If you think data-driven approaches…

I definitely saw somebody at Actuate last year talking about supplementing training videos for VLA with Youtube, but I think they actually found that "any" video of the real world helped give a better physics "understanding" to the model.

Re: We accidentally solved robotics by watching 1M hours of YouTube

#105

Post by carlosdp »

dchftcs wrote: post #78

Pure vision will never be enough because it does not contain information about the physical feedback like pressure and touch, or the strength required to perform a task. For example, so that you don't crush a human when doing massage (but still need to press hard), or apply the right amount of force (and finesse?) to skin a fish fillet without cutting the skin itself. Practically in the near term, it's hard to sample…

> Pure vision will never be enough because it does not contain information about the physical feedback like pressure and touch, or the strength required to perform a task.

I'm not sure that's necessarily true for a lot of tasks.

A good way to measure this in your head is this:

"If you were given remote control of two robot arms, and just one camera to look through, how many different tasks do you think you could complete successfully?"

When you start thinking about it, you realize there are a lot of things you could do with just the arms and one camera, because you as a human have really good intuition about the world.

It therefore follows that robots should be able to learn with just RGB images too! Counterexamples would be things like grabbing an egg without crushing, perhaps. Though I suspect that could also be done with just vision.

Re: We accidentally solved robotics by watching 1M hours of YouTube

#106

Post by marcus_holmes »

shadowgovt wrote: post #72
marcus_holmes wrote: post #66

Earlier quoted context omitted.

There's an English phrase "hounded to death", meaning that someone was pursued and hassled until they died. It doesn't specify the cause of death, but I think the assumption would be suicide, since you can't actually die of fatigue. I think that's what was done to Aaron Swartz.

Many people have dealt with the law, with copyright infringement, even with gross amounts of it, and had the book thrown at them, and survived the experience. Swartz was ill. It is a tragedy he did not survive the experience, and indeed, trial is very stressful. But he was no more hounded than any defendant who comes under federal scrutiny and has to defend themselves in a court of law via the trial system. Kevin Mit…

Many people get (for example) pneumonia and recover. Some people get pneumonia and die. The people who died of pneumonia died because of pneumonia. The fact that other people survived it doesn't mean that they didn't die of it.

Saying that we should not work on cures for pneumonia because it's a Chesterton Fence is obviously, blatantly, illogical. Saying that we should change the system so that government officials working for moneyed interests can't hound someone to death is similarly illogical.

Re: We accidentally solved robotics by watching 1M hours of YouTube

#107

Post by rzzzt »

washadjeffmad wrote: post #58
isoprophlex wrote: post #4

Earlier quoted context omitted.

How much is that in football fields?

Good catch. Approximately 9,192,631 Turkish decibels.

Fun fact: the International Bureau of Weights and Measures in Paris is the owner of a perfect 0 dB noise floor enclosed in a perfect titanium sphere (with some sheep's wool filling to avoid reflections). There is a small door on the side over which microphone capsules can be inserted for calibration.

(/joke)

Re: We accidentally solved robotics by watching 1M hours of YouTube

#108

Post by seydor »

column wrote: post #103

> right now, you need to show the robot pictures of what you want it to do. want it to "clean the kitchen"? better have a photo of a clean kitchen handy. What about using Flux Kontext (or Controlnets) to turn the messy kitchen into a clean kitchen?

Sure thing, let me just put the fridge in the washing machine.

Re: We accidentally solved robotics by watching 1M hours of YouTube

#109

Post by jpc0 »

carlosdp wrote: post #105
dchftcs wrote: post #78

Pure vision will never be enough because it does not contain information about the physical feedback like pressure and touch, or the strength required to perform a task. For example, so that you don't crush a human when doing massage (but still need to press hard), or apply the right amount of force (and finesse?) to skin a fish fillet without cutting the skin itself. Practically in the near term, it's hard to sample…

> Pure vision will never be enough because it does not contain information about the physical feedback like pressure and touch, or the strength required to perform a task. I'm not sure that's necessarily true for a lot of tasks. A good way to measure this in your head is this: "If you were given remote control of two robot arms, and just one camera to look through, how many different tasks do you think you could comp…

I think you vastly underestimate how difficult the task you are proposing would be without depth or pressure indication, even for a super intelligence like humans.

Simple concept, pick up a glass and pour its content into a vertical hole the approximate size of your mouth. Think of all the failure modes that can be triggered in the trivial example you do multiple times a day, to do the same from a single camera feed with no other indicators would take you hours to master and you already are a super intelligent being.

Re: We accidentally solved robotics by watching 1M hours of YouTube

#110

Post by moefh »

carlosdp wrote: post #105
dchftcs wrote: post #78

Pure vision will never be enough because it does not contain information about the physical feedback like pressure and touch, or the strength required to perform a task. For example, so that you don't crush a human when doing massage (but still need to press hard), or apply the right amount of force (and finesse?) to skin a fish fillet without cutting the skin itself. Practically in the near term, it's hard to sample…

> Pure vision will never be enough because it does not contain information about the physical feedback like pressure and touch, or the strength required to perform a task. I'm not sure that's necessarily true for a lot of tasks. A good way to measure this in your head is this: "If you were given remote control of two robot arms, and just one camera to look through, how many different tasks do you think you could comp…

> It therefore follows that robots should be able to learn with just RGB images too!

I don't see how that follows. Humans have trained by experimenting with actually manipulating things, not just by vision. It's not clear at all that someone who had gained intuition about the world exclusively by looking at it would have any success with mechanical arms.

Post reply on HN