Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SAM 2: Segment Anything in Images and Videos

github.com

Post reply on HN
Print view View original

71–80 of 159 posts

Re: SAM 2: Segment Anything in Images and Videos

#71

Post by Legend2440 »

madduci wrote: post #65
nravi20 wrote: post #2

Hi from the Segment Anything team! Today we’re releasing Segment Anything Model 2! It's the first unified model for real-time promptable object segmentation in images and videos! We're releasing the code, models, dataset, research paper and a demo! We're excited to see what everyone builds! https://ai.meta.com/blog/segment-anything-2/

Thank you for sharing it! Is there any plans to move the codebase to a more performant programming language?

Everything in machine learning uses Python.

It doesn't matter much because all the real computation happens on the GPU. But you could take their neural network and do inference using any language you want.

Re: SAM 2: Segment Anything in Images and Videos

#72

Post by bryanrasmussen »

j7ake wrote: post #52
vivzkestrel wrote: post #45

Earlier quoted context omitted.

stupid question from a noob: what exactly is object segmentation? what does your library actually do? Does it cut clips?

Given an image, it will outline where objects are in the image.

and extract segments of images where the object are in the image as I understand it?

A segment then is a collection of images that follow each other in time?

So if you have a video comprised of img1, img2, img3, img4 and object shows in img1 and img2 and img4

Can you catch that as a sequence img1, img2, img3, img4 and can you also catch just the object img1, img2, img4 but get some sort of information that there is a break between img2 and img4 - number of images break etc.?

On edit: Or am I totally off about the segment possibilities and what it means?

Or can you only catch img1 and img2 as a sequence?

Re: SAM 2: Segment Anything in Images and Videos

#73

Post by phillypham »

kajecounterhack wrote: post #70
phillypham wrote: post #57

Really cool. Doesn't really work for juggling unfortunately, https://sam2.metademolab.com/shared/fa993f12-b9ce-4f19-bb75-...

It looks like it’s working to me. Segmentation isn’t supposed to be used for tracking alone. If you add tracking on top, the uncertainty in the estimated mask for the white ball (which is sometimes getting confused with the wall) would be accounted for and you’d be able to track it well.

The blog post (https://ai.meta.com/blog/segment-anything-2/) mentions tracking as a use case. Similar objects is known to be challenging and they mention it in the Limitations section. In that video, I only used one frame, but in some other tests even when I prompted in several frames as recommended, it didn't really work, still.

Re: SAM 2: Segment Anything in Images and Videos

#74

Post by stabbles »

vivzkestrel wrote: post #45
nravi20 wrote: post #2

Hi from the Segment Anything team! Today we’re releasing Segment Anything Model 2! It's the first unified model for real-time promptable object segmentation in images and videos! We're releasing the code, models, dataset, research paper and a demo! We're excited to see what everyone builds! https://ai.meta.com/blog/segment-anything-2/

stupid question from a noob: what exactly is object segmentation? what does your library actually do? Does it cut clips?

Classification per pixel

Re: SAM 2: Segment Anything in Images and Videos

#75

Post by sashank_1509 »

rkagerer wrote: post #53
minimaxir wrote: post #14

The web demo is actually pretty neat: https://sam2.metademolab.com/demo I selected each shoe as individual objects and the model was able to segment them even as they overlapped.

I guess the demo simply doesn't work unless you accept cookies?

Are there people who don’t accept cookies?

Don’t most websites require you to accept cookies?

Re: SAM 2: Segment Anything in Images and Videos

#77

Post by doubleorseven »

Thank you for this amazing work you are sharing.

I do have a 2 questions: 1. isn't addressing the video frame by frame expensive? 2. In the web demo when the leg moves fast it loses it's track from the shoe. Does the memory part not throwing some uristics to over come this edge case?

Re: SAM 2: Segment Anything in Images and Videos

#78

Post by shreddit »

sashank_1509 wrote: post #75
rkagerer wrote: post #53

Earlier quoted context omitted.

I guess the demo simply doesn't work unless you accept cookies?

Are there people who don’t accept cookies? Don’t most websites require you to accept cookies?

If someone gives me the choice i don’t.

Re: SAM 2: Segment Anything in Images and Videos

#80

Post by pgt »

Wonder if I can use this to count my winter wood stock. Before resuscitating my mutilated Python environment, could someone please run this on a photo of stacked uneven bluegum logs to see if it can segment the pieces? OpenCV edge detection does not cut it:

https://share.icloud.com/photos/090J8n36FAd0_lz4tz-TJfOhw

Post reply on HN