Meta's Segment Anything written with C++ / GGML • Hacker News Board

Re: Meta's Segment Anything written with C++ / GGML

#31

Post by shepardrtc » Thu, Sep 07, 2023, 12:39 AM UTC

dbmikus wrote: post #30

IshKebab wrote: post #28

Earlier quoted context omitted.

Setting up and deploying models in production or on edge devices is much much more complex if you have to deal with Python and Conda and whatnot.

You can compile the models to something that runs on edge though, right? For example, Tensorflow is a C++ framework that has Python bindings and a Python library, but when the models are served they are running on C++. Maybe the act of compilation is an extra step, but I'd much rather have my development be in a high level language that is very suited to experimentation, probing, and testing, and then compile the fin…

Cloudflare lets you just upload the model itself: https://blog.cloudflare.com/introducing-constellation/

No idea what they're using to run it though. But there's no way I'll stop using Python for working with ML code lol. It just makes life easy.

Re: Meta's Segment Anything written with C++ / GGML

#32

Post by Tostino » Thu, Sep 07, 2023, 3:26 PM UTC

I am looking for a model similar to this, but for text. I want to group text with different labels that apply to subsets of the text. Think of being able to quickly pull-out related segments from a large body of text. Let's take, for instance, a sales contract that specifies a discounted price for various goods. If you select the label "data rows", the system should be able to extract all the text pertaining to the table that specifies which SKUs are being purchased, and at what discounted price. Moreover, this model should be capable of segmenting the content into semantically relevant chunks. One example: each row in the aforementioned table would be tagged with multiple labels. One would be just that it is a row, the data in the first column should be labeled for what it represents, e.g. "product number". Another example: if there's a section discussing the terms of delivery or warranty conditions, selecting the respective labels would instantly extract that specific information, regardless of where it's located within the document. Would be great for it to be able to segment into some controllable range of tokens/characters to allow for pulling those chunks into a vector database, along with the relevant tags related to the chunk.

Re: Meta's Segment Anything written with C++ / GGML

#33

Post by IshKebab » Thu, Sep 07, 2023, 6:25 PM UTC

dbmikus wrote: post #30

IshKebab wrote: post #28

Earlier quoted context omitted.

Setting up and deploying models in production or on edge devices is much much more complex if you have to deal with Python and Conda and whatnot.

You can compile the models to something that runs on edge though, right? For example, Tensorflow is a C++ framework that has Python bindings and a Python library, but when the models are served they are running on C++. Maybe the act of compilation is an extra step, but I'd much rather have my development be in a high level language that is very suited to experimentation, probing, and testing, and then compile the fin…

> Tensorflow is a C++ framework that has Python bindings and a Python library, but when the models are served they are running on C++

Sure, and it's only a simple 20 step process that involves building Tensorflow from source. Yeay!

https://medium.com/@hamedmp/exporting-trained-tensorflow-mod...

Let me see what the process for compiling a LLM written in Rust is....

https://github.com/rustformers/llm

    cargo install llm-cli

Oh look, it doesn't make me immediately want to give up.

Re: Meta's Segment Anything written with C++ / GGML

#34

Post by dbmikus » Thu, Sep 07, 2023, 6:53 PM UTC

IshKebab wrote: post #33

dbmikus wrote: post #30

Earlier quoted context omitted.

You can compile the models to something that runs on edge though, right? For example, Tensorflow is a C++ framework that has Python bindings and a Python library, but when the models are served they are running on C++. Maybe the act of compilation is an extra step, but I'd much rather have my development be in a high level language that is very suited to experimentation, probing, and testing, and then compile the fin…

> Tensorflow is a C++ framework that has Python bindings and a Python library, but when the models are served they are running on C++ Sure, and it's only a simple 20 step process that involves building Tensorflow from source. Yeay! https://medium.com/@hamedmp/exporting-trained-tensorflow-mod... Let me see what the process for compiling a LLM written in Rust is.... https://github.com/rustformers/llm cargo install llm-…

llm-cli looks like it loads model files and it doesn't help with model development. It runs GGML model files. The models aren't written in Rust. Besides the point, GGUF is successor to GGML. There's a variety of ways to convert Pytorch, Keras, etc models to GGML or GGUF.

I dunno, maybe we're talking about different things. I'm saying it's better to do model development in a high level language and then export the training or runtime to a lower level framework, multiple of which exist and have existed. It's becoming simpler to use low-level runtimes (llama.cpp vs Tensorflow). Is that the point you're making?