Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SAM 2: Segment Anything in Images and Videos

github.com

Post reply on HN
Print view View original

61–70 of 159 posts

Re: SAM 2: Segment Anything in Images and Videos

#62

Post by 8organicbits »

robbomacrae wrote: post #4
nravi20 wrote: post #2

Hi from the Segment Anything team! Today we’re releasing Segment Anything Model 2! It's the first unified model for real-time promptable object segmentation in images and videos! We're releasing the code, models, dataset, research paper and a demo! We're excited to see what everyone builds! https://ai.meta.com/blog/segment-anything-2/

Code, model, data and under Apache 2.0. Impressive. Curious how this was allowed to be more open source compared to Llama's interesting new take on "open source". Are other projects restricted in some form due to technical/legal issues and the desire is to be more like this project? Or was there an initiative to break the mold this time round?

Yeah, but there's a CLA for some reason. I'm wary they will switch to a new license down the road.

Re: SAM 2: Segment Anything in Images and Videos

#65

Post by madduci »

nravi20 wrote: post #2

Hi from the Segment Anything team! Today we’re releasing Segment Anything Model 2! It's the first unified model for real-time promptable object segmentation in images and videos! We're releasing the code, models, dataset, research paper and a demo! We're excited to see what everyone builds! https://ai.meta.com/blog/segment-anything-2/

Thank you for sharing it! Is there any plans to move the codebase to a more performant programming language?

Re: SAM 2: Segment Anything in Images and Videos

#67

Post by glandium »

RobinL wrote: post #64
glandium wrote: post #63

> We extend SAM to video by considering images as a video with a single frame. I can't make sense of this sentence. Is there some mistake?

Everything is a video. An image is the special case of length 1 frame

Here's a sentence I would understand: > We extend SAM to video and retrofit support for images by considering images as a video with a single frame.

As it is written, I don't see the link between "We extend SAM to video" and "by considering images as a video with a single frame".

Re: SAM 2: Segment Anything in Images and Videos

#68

Post by dingaling »

alsodumb wrote: post #31
swyx wrote: post #23

i covered SAM 1 a year ago ( https://news.ycombinator.com/item?id=35558522 ). notes from quick read of the SAM 2 paper https://ai.meta.com/research/publications/sam-2-segment-anyt... 1. SAM 2 was trained on 256 A100 GPUs for 108 hours (SAM1 was 68 hrs on same cluster). Taking the upper end $2 A100 cost off gpulist means SAM2 cost ~$50k to train - surprisingly cheap for adding video understanding? 2. new dataset: the…

I might be minority, but I am not that surprised by the results or the not so significant GPU hours. I've been video segment tracking for a while now using SAM for mask generation and some of the robust academic video-object segmentation models (see CUTIE: https://hkchengrex.com/Cutie/ presented at CVPR this year.)for tracking the mask. I need to read SAM2 paper, but 4. seems a lot like what Rex has in CUTIE. CUTIE c…

Seems like there's functional overlap between segmentation models and the autofocus algorithms developed by Canon and Sony for their high-end cameras.

The Canon R1 for example will not only continually track a particular object even if partially occluded but will also pre-focus on where it predicts the object will be when it emerged from being totally hidden. It can also be programmed by the user to focus on a particular face to the exclusion of all else.

Re: SAM 2: Segment Anything in Images and Videos

#70

Post by kajecounterhack »

phillypham wrote: post #57

Really cool. Doesn't really work for juggling unfortunately, https://sam2.metademolab.com/shared/fa993f12-b9ce-4f19-bb75-...

It looks like it’s working to me. Segmentation isn’t supposed to be used for tracking alone. If you add tracking on top, the uncertainty in the estimated mask for the white ball (which is sometimes getting confused with the wall) would be accounted for and you’d be able to track it well.
Post reply on HN