Live data from Hacker News

TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

github.com

31–40 of 43 posts

Re: TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

#31

This is cool. I wonder with stuff like this, what happens if a self driving car is capable of processing reflections in glass windows? What if it sees a reflection of itself and is able to properly identify it as being itself? Does that make it self aware? I'm being serious. People like to throw around terms like "self aware" with some assumption that it is a long way off, or impossible, to have a machine be self awa…

Many robots already have cameras pointed at themselves. What does that change really?

Honestly nothing, to me. But other people think that ability to recognize oneself in a mirror is important.

To me the concept of self-awareness and consciousness is pretty much meaningless, especially if you are considering it something that machines don't have or can't have (or if they eventually do have it, we'll know).

The reason I mention it with this, and with self driving cars (which this particular system may not be fast or reliable for yet), is that for those people, it may register better because it seems more analogous to a human. With those robots you speak of, do they also recognize other robots? Do they have some sort of logic that knows that they are like those other robots in many ways, but in significant ways they are different (i.e. they have control over their own behavior but not over the others')?

The point is not that something particularly amazing is happening, the point is that it is getting easier to illustrate with real world examples that "self awareness" is not this magical thing we currently have no idea how it works.

Re: TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

#32

Can someone add depth to the training set? I'd like depth estimates for objects in the frame too. It could be interesting to fly into a video. Does the iPhone 8 have rgbd now for short range? Maybe someday we could get pixel by pixel depth estimates?

This exists in several different implementations and would be a separate DNN.

http://cs231n.stanford.edu/reports/2017/pdfs/203.pdf

Re: TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

#33

Earlier quoted context omitted.

(1) It's not free when the research is funded by public tax dollars (which I grant is not always, but often the case). (2) I'm not aware of this promise you're speaking of... my understanding is that authors of papers are under no obligation to produce any implementation, let alone usable, documented implementations.

> (1) It's not free when the research is funded by public tax dollars (which I grant is not always, but often the case). The work was done at FAIR (Facebook), so that's not applicable here. > (2) I'm not aware of this promise you're speaking of... my understanding is that authors of papers are under no obligation to produce any implementation, let alone usable, documented implementations. Look at the paper [1], it's…

They said at ICCV that they will release the code after the CVPR paper submission deadline, so "soon".

Re: TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

#34

Can someone add depth to the training set? I'd like depth estimates for objects in the frame too. It could be interesting to fly into a video. Does the iPhone 8 have rgbd now for short range? Maybe someday we could get pixel by pixel depth estimates?

The iPhone 7+, 8+, and X running iOS 11 have the ability to provide interpolated pixel depth maps for portrait photos, although it’s a fairly rough approximation. More details at https://developer.apple.com/documentation/avfoundation/avdep.... There’s also a really good video from WWDC this year that drills into it.

Re: TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

#35
post #10

Earlier quoted context omitted.

Deepmask is something else.

You are right, it's a different paper. But they both do instance segmentation and both build upon faster r cnn.

They do not both build upon faster-rcnn. Deep mask is a pretty different paradigm - it makes mask proposals (instead of bounding box proposals) and then refines them using sharpmask, and it doesn't work as well as mask-rcnn in terms of mean average precision. Mask-rcnn just adds a branch to faster-rcnn that roi-pools the bounding box predictions in order to predict masks.

Re: TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

#36
post #21

Earlier quoted context omitted.

[edit] > people who implement models from academic articles Is anyone collecting the various HN discussions as these pop up? I would appreciate help finding them again.

If you're looking for a way to find the implementations again I recommend http://www.gitxiv.com/

this should be the standard way for publishing

Re: TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

#37
post #17

Very cool! How is the performance? R-CNN used to be much slower comparing to YOLO or SSD; FCN seems to be very fast as well though requires a lot of GPU memory. Can that your version be used for realtime semantic segmentation?

This architecture is optimized for accuracy rather than speed. The official paper reports 200ms inferencing time per image on a GPU. This implementation is likely a bit slower because we use Python in a couple of layers. This is easy to optimize, but we haven't gotten around to it yet.

With that said, there are a lot of things you could do to make this much faster. For example, use ResNet50 instead of ResNet101. You can also reduce the number of anchors or the number of proposals to classify, and that should improve performance significantly at the expense of a little loss in accuracy.

Re: TensorFlow Mask R-CNN code for pixelwise object detection and segmentation

#39

Wow, this is fascinating. From a radiology perspective, it could be the missing method for segmenting findings inside a convoluted radiograph.

Yes, and that's possible now with many different CNNs. The limiting factor is the training/validation/test data in your subject. For example you couldn't implement Mask R-CNN with the COCO dataset as implemented here and get inference on your radiology problem set.

Would transfer learning help with that?
Post reply on HN