Hmm wonder how this compares to stuff like FastSAM and MobileSAM. Is SAM quantized better or are those knock of architectures more performant.
Meta's Segment Anything written with C++ / GGML
21–30 of 34 posts
Re: Meta's Segment Anything written with C++ / GGML
#22I'm so glad the AI community is finally starting to ditch Python. It has held progress back for far too long.
This is exactly the wrong way around. We've seen the progress we've seen because of the adoption of Python. Even now there are relatively few people that can write code like this and have the ML and math experience to push forward the research.
Re: Meta's Segment Anything written with C++ / GGML
#23While I love the efficiency from these Python to C++ ports I can't stop thinking about the long tail of subtle bugs that will likely infest these libraries forever but then the Python versions also sit atop C/C++ cores
Just wait until they’re ported to C++ using AI!
Re: Meta's Segment Anything written with C++ / GGML
#24This is a port of Meta's Segment Anything computer vision model which allows easy segmentation of shapes in images. Originally written in Python, Yavor Ivanov has ported it to C++ using the GGML library created by Georgi Gerganov which is optimized for CPU instead of GPU, specifically Apple Silicon M1/M2. The repo is still in it's early stage
Do you know how the time to do the image embedding takes? In SAM, most of the time is spent generating a very expensive embedding (prohibitive for real-time object detection). From the timing on your page it looks like yours is also similarly slow, but I'm curious how it compares to the pytorch Meta implementation.
I am the creator of the repo.
Depends on the machine, number of threads selected and the model checkpoint used (Vit-B or Vit-L or Vit-B). The video demo attached is running on Apple M2 Ultra and using the Vit-B model. The generation of the image embedding takes ~1.9s there and all the subsequent mask segmentations take ~45ms.
However, I am now focusing on improving the inference speed by making better use of ggml and trying out quantization. Once I make some progress in this direction I will compare to other SAM alternatives and benchmark more thoroughly.
Re: Meta's Segment Anything written with C++ / GGML
#25Earlier quoted context omitted.
Do you know how the time to do the image embedding takes? In SAM, most of the time is spent generating a very expensive embedding (prohibitive for real-time object detection). From the timing on your page it looks like yours is also similarly slow, but I'm curious how it compares to the pytorch Meta implementation.
I am the creator of the repo. Depends on the machine, number of threads selected and the model checkpoint used (Vit-B or Vit-L or Vit-B). The video demo attached is running on Apple M2 Ultra and using the Vit-B model. The generation of the image embedding takes ~1.9s there and all the subsequent mask segmentations take ~45ms. However, I am now focusing on improving the inference speed by making better use of ggml and…
This is amazing. Thank you!
Re: Meta's Segment Anything written with C++ / GGML
#26Bravo, the demonstration is genuinely impressive!
Next Step: Incorporate this library into image editors like Photopea (via WebAssembly) to boost the speed of common selection tasks. The magic wand is a tool of the past.
I'd pay for such a feature.
Re: Meta's Segment Anything written with C++ / GGML
#27I'm so glad the AI community is finally starting to ditch Python. It has held progress back for far too long.
The AI community is nowhere close to ditching Python. Most model development and training still use python based toolchains (torch, tf...). The new trends is for popular and useful models to be ported to more efficient stack like C++/GGML for easier usage and inference speed on consumer hardware. Another popular optimisation is to port models to WASM + GPU because it makes them easy to support a variety of platforms…
That's why I said "starting to" not "close to".
Re: Meta's Segment Anything written with C++ / GGML
#28I'm so glad the AI community is finally starting to ditch Python. It has held progress back for far too long.
How has Python held it back? Most of the heavy computation lifting is done by C extensions/bindings and the models are compiled to run on CUDA, etc. What am I missing?
Setting up and deploying models in production or on edge devices is much much more complex if you have to deal with Python and Conda and whatnot.
Re: Meta's Segment Anything written with C++ / GGML
#29Earlier quoted context omitted.
How has Python held it back? Most of the heavy computation lifting is done by C extensions/bindings and the models are compiled to run on CUDA, etc. What am I missing?
Presumably what you're missing is that it's that IshKebab probably doesn't work in AI/ML at all (no links in his profile, but you can judge his post history yourself). Anyone can have voice opinion, but that doesn't mean it's particularly well informed.
I worked for an AI startup for 5 years until recently. Nice try though.
Re: Meta's Segment Anything written with C++ / GGML
#30Earlier quoted context omitted.
How has Python held it back? Most of the heavy computation lifting is done by C extensions/bindings and the models are compiled to run on CUDA, etc. What am I missing?
Setting up and deploying models in production or on edge devices is much much more complex if you have to deal with Python and Conda and whatnot.
You can compile the models to something that runs on edge though, right? For example, Tensorflow is a C++ framework that has Python bindings and a Python library, but when the models are served they are running on C++.
Maybe the act of compilation is an extra step, but I'd much rather have my development be in a high level language that is very suited to experimentation, probing, and testing, and then compile the final result down to something performant.
EDIT: I don't know much about the IOT world, and Tensorflow is likely a bad example as it's not designed to run on edge. So, I could understand that things like llama.cpp, GGML and GGUF are making strides towards easier runtimes. But I still think for dev-time, Python makes sense!