Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teaching Robots to Understand Semantic Concepts

research.googleblog.com

Post reply on HN
Print view View original

11–20 of 20 posts

Re: Teaching Robots to Understand Semantic Concepts

#11

Post by ericjang »

skywhopper wrote: post #9

The headline seems to misuse the word "semantic" (not to mention "understand"). Does the door-opening robot now understand how to open all hinged doors with a similar opening mechanism? Or was it just trained to imitate a sequence of changes in a 2D image from a fixed angle? Can the same software and robot also be taught to open windows? Boxes? We are talking about "semantics" explicitly here. Does it understand "ope…

Interesting points. What would you consider an adequate task that demonstrates a program/robot has acquired "semantic understanding"?

Re: Teaching Robots to Understand Semantic Concepts

#12

Post by briga »

ragebol wrote: post #8
QAPereo wrote: post #2

To reference an earlier article on HN, this reads like a future of, "Robot, observe these field workers picking fruit for a week. Now practice in this field for a day. OK, now the job is yours. What I can't tell at all from this article is whether that day is years or decades away.

https://xkcd.com/1425/ It reall, really depends on the job. And in a job, maybe some subtasks can be automated. Perhaps the picking cannot be automated, but the transportation can. Or instead of computer vision, have a person click the fruit on a screen so the robot know where to pick.

Crazy to think that something that was considered "virtually impossible" (image recognition) a few years ago is relatively easy today.

Re: Teaching Robots to Understand Semantic Concepts

#13

Post by webmaven »

Swizec wrote: post #5
QAPereo wrote: post #2

To reference an earlier article on HN, this reads like a future of, "Robot, observe these field workers picking fruit for a week. Now practice in this field for a day. OK, now the job is yours. What I can't tell at all from this article is whether that day is years or decades away.

> whether that day is years or decades away Yes, it's both. Depends on the task. Afaik we can already program industrial robots by showing them what to do. Robot records movement in its actuators, then keeps replaying over and over. And we already have machine learning agents that can observe your behavior and learn which news stories are "good" news stories and which aren't. (algo newsfeeds). You could use them to o…

That's not "showing them what to do" it is more along the lines of "hand-holding".

Re: Teaching Robots to Understand Semantic Concepts

#14

Post by Vivian99 »

suyash wrote: post #10
skywhopper wrote: post #9

The headline seems to misuse the word "semantic" (not to mention "understand"). Does the door-opening robot now understand how to open all hinged doors with a similar opening mechanism? Or was it just trained to imitate a sequence of changes in a 2D image from a fixed angle? Can the same software and robot also be taught to open windows? Boxes? We are talking about "semantics" explicitly here. Does it understand "ope…

That seems to be what they are aiming towards but true AI is not the same as Machine Learning. Google is still using what we could call a very rudimentary form of AI as they describe "Unsupervised learning on very small datasets is one of the most challenging scenarios in machine learning. To make this feasible, we use deep visual features from a large network trained for image recognition on ImageNet".

Since I started with my online business I earn $95 every 15 minutes. It sounds unbelievable but you wont forgive yourself if you don't check it out.

----------------->>http://www.joinmate2.Com

Re: Teaching Robots to Understand Semantic Concepts

#15

Post by Houshalter »

skywhopper wrote: post #9

The headline seems to misuse the word "semantic" (not to mention "understand"). Does the door-opening robot now understand how to open all hinged doors with a similar opening mechanism? Or was it just trained to imitate a sequence of changes in a 2D image from a fixed angle? Can the same software and robot also be taught to open windows? Boxes? We are talking about "semantics" explicitly here. Does it understand "ope…

I tied a rope to my door handle and put a treat on the other side of the door. I tried for half an hour to get my dog to learn how to open the door. I demonstrated it many times. I guided his motions through the procedure. I put a treat on the rope to encourage him to interact with it. He moved the door chewing on the treat, but still didn't learn how to open the door on purpose. I gave up.

So yeah maybe it's not anywhere near human intelligence. But its still cool they've made a robot smarter than my dog.

Re: Teaching Robots to Understand Semantic Concepts

#16

Post by Hadley434 »

suyash wrote: post #10
skywhopper wrote: post #9

The headline seems to misuse the word "semantic" (not to mention "understand"). Does the door-opening robot now understand how to open all hinged doors with a similar opening mechanism? Or was it just trained to imitate a sequence of changes in a 2D image from a fixed angle? Can the same software and robot also be taught to open windows? Boxes? We are talking about "semantics" explicitly here. Does it understand "ope…

That seems to be what they are aiming towards but true AI is not the same as Machine Learning. Google is still using what we could call a very rudimentary form of AI as they describe "Unsupervised learning on very small datasets is one of the most challenging scenarios in machine learning. To make this feasible, we use deep visual features from a large network trained for image recognition on ImageNet".

Start making more money weekly.This is a valuable part time work for everyone.The best part work from comfort of your house and get paid from $100-$2k each week. Start today and have your first cash at the end of this week....>http://ow.ly/iBXm30dNtIZ

Re: Teaching Robots to Understand Semantic Concepts

#17

Post by joshvm »

QAPereo wrote: post #2

To reference an earlier article on HN, this reads like a future of, "Robot, observe these field workers picking fruit for a week. Now practice in this field for a day. OK, now the job is yours. What I can't tell at all from this article is whether that day is years or decades away.

This is unlikely to be how the fruit pickers of the future work (though I get your point). Using a robotic arm like this is an extremely expensive way of solving the problem. That's ignoring the technical challenges like accurate localisation of fruit in small clusters, not damaging crop, dealing with occlusions from leaves, etc.

My personal opinion, having worked a little on this problem, is that it's very much like autonomous driving. Getting 90% of the job done is fairly straightforward, but getting that final 10% to make a system commercially viable will take years. Commercial growers don't (yet) have any pressure on labour - it's too readily available and too cheap.

http://www.agrobot.com/ solves this in quite a neat way. Rather than grasping the fruit, they just scoop up each berry in a cup with a blade on one side which severs the stem. This means you don't care about the precise shape of the berry either.

Re: Teaching Robots to Understand Semantic Concepts

#18

Post by richard___ »

Summary--

Semantics part: Seems like the idea is we can "transfer" knowledge from prior labeled samples so that we don't need to do as much new work labeling sample images with semantic labels.

Grasping part: "Emulating human movements with self-supervision and imitation." High-level imitation based on visual frame differences avoids needing to manually control actuators. Not sure how this works exactly

Two-stream model: ventral network asks What class, dorsal network asks Is this how we should grasp this object. The benefit is that we can make use of all the automatically generated (robot-generated) grasping data without having a human supervise all that automated grasping, e.g. "This process is a successful way to pickup this object, and also this object is an apple." The ventral network ties back this the grasping data (without object labels) to object labels, which allows for semantic control of the trained robot e.g. "Pickup that apple".

Re: Teaching Robots to Understand Semantic Concepts

#19

Post by miltondts »

Houshalter wrote: post #15
skywhopper wrote: post #9

The headline seems to misuse the word "semantic" (not to mention "understand"). Does the door-opening robot now understand how to open all hinged doors with a similar opening mechanism? Or was it just trained to imitate a sequence of changes in a 2D image from a fixed angle? Can the same software and robot also be taught to open windows? Boxes? We are talking about "semantics" explicitly here. Does it understand "ope…

I tied a rope to my door handle and put a treat on the other side of the door. I tried for half an hour to get my dog to learn how to open the door. I demonstrated it many times. I guided his motions through the procedure. I put a treat on the rope to encourage him to interact with it. He moved the door chewing on the treat, but still didn't learn how to open the door on purpose. I gave up. So yeah maybe it's not any…

I think this says more about your lack of knowledge. See https://www.youtube.com/watch?v=QKSvu3mj-14 for more information.

EDIT: To provide a bit more information. I provide a video because I think it is much more impactful to see what happens than to read it. In this video you can see how an appropriate history of reinforcement will lead to very complex behavior in simple animals. By complex I mean behavior like "talking" and "problem solving".

Here is the 2nd part: https://www.youtube.com/watch?v=erhmslcHvaw

Re: Teaching Robots to Understand Semantic Concepts

#20

Post by tanilama »

ericjang wrote: post #11
skywhopper wrote: post #9

The headline seems to misuse the word "semantic" (not to mention "understand"). Does the door-opening robot now understand how to open all hinged doors with a similar opening mechanism? Or was it just trained to imitate a sequence of changes in a 2D image from a fixed angle? Can the same software and robot also be taught to open windows? Boxes? We are talking about "semantics" explicitly here. Does it understand "ope…

Interesting points. What would you consider an adequate task that demonstrates a program/robot has acquired "semantic understanding"?

Understand instruction in natural language.

For example:

1.'grab that red ball'

2.'turn the handle on the door 90 degree then pull it out'.

Just like how people would do it, a video and a piece of instruction listed and a label to indicate whether this task is a success or not. Then you show a different setting and a new instruction, if the model successfully generalize and understand the semantics behind it, it should carry out the instruction successfully.

Post reply on HN