Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fuyu-8B: A multimodal architecture for AI agents

adept.ai

Post reply on HN
Print view View original

41–50 of 60 posts

Re: Fuyu-8B: A multimodal architecture for AI agents

#41

Post by coder543 »

abrichr wrote: post #40
coder543 wrote: post #37

Earlier quoted context omitted.

"fully open source", but there is no license? https://github.com/OpenAdaptAI/OpenAdapt/blob/30581e47fa9aec... https://github.com/OpenAdaptAI/OpenAdapt/issues/246 And Fuyu is under a non-commercial license, so there's not much to be done with it unless someone trains a new Fuyu-architecture model from scratch.

Thank you for pointing this out! You are correct that we have not yet decided on a license. I will admit my ignorance on this topic, and I didn't want us to rush into selecting a license that is inappropriate. Which one should we choose?

> Which one should we choose?

It depends a lot on what you want the license to do, so I don’t really want to say one way or another.

IANAL, but my understanding is that code without a license effectively has an “all rights reserved” license in the U.S., meaning that it can’t be used for anything at all — even non-commercial work.

Re: Fuyu-8B: A multimodal architecture for AI agents

#42

Post by brianjking »

tasdfqwer0897 wrote: post #2

Hey I work at Adept and helped make this! Happy to answer questions. The thing I think is especially neat/notable is how simple you can make the model architecture while still getting good performance. I expect we'll continue to see bits of these models get deleted in the next few years Note that you can get the model weights on HuggingFace here: https://huggingface.co/adept/fuyu-8b

First off, absolutely incredible work, congrats and thank you.

Secondly, do you anticipate Fuyu being made available for commercial access or will it remain NC?

Re: Fuyu-8B: A multimodal architecture for AI agents

#43

Post by startupsfail »

outofpaper wrote: post #36
MattPalmer1086 wrote: post #23

Earlier quoted context omitted.

IANAL but as far as I remember, you can't copyright a list of objective facts, for example a phone book containing a list of phone numbers. Model weights are clearly not in that category. Happy to be corrected if I misremember.

Model weight are akin to markov chains and compressed data. They are direct representations of the data they where created from in the same way that markov chains are created from hidden markov chains and Zipped files are created from files. Zipping a file does not grant the copyright protection of the zipped output beyond the copyright of the original file. Moreover the American federal registrar has officially stat…

If you take some copyrighted data, a set of books, for example. And count words in these books and then plot a distribution of top 100 word frequencies. The copyright for that new image would belong to you.

Re: Fuyu-8B: A multimodal architecture for AI agents

#44

Post by webappguy »

abrichr wrote: post #40
coder543 wrote: post #37

Earlier quoted context omitted.

"fully open source", but there is no license? https://github.com/OpenAdaptAI/OpenAdapt/blob/30581e47fa9aec... https://github.com/OpenAdaptAI/OpenAdapt/issues/246 And Fuyu is under a non-commercial license, so there's not much to be done with it unless someone trains a new Fuyu-architecture model from scratch.

Thank you for pointing this out! You are correct that we have not yet decided on a license. I will admit my ignorance on this topic, and I didn't want us to rush into selecting a license that is inappropriate. Which one should we choose?

If it's for the win (?), the most permissible is the one you choose. This is a extraordinarily competitive space. The sooner you make the choice and it's MIT, the sooner I personally put forth serious contribution time and the faster you grow in the broad and competitive ecosystem. Your main options are GNU All-permissive License, MIT License, BSD licenses, Apple Public Source License and Apache license.

It is recommended by this developer you go MIT

Re: Fuyu-8B: A multimodal architecture for AI agents

#45

Post by abrichr »

webappguy wrote: post #44
abrichr wrote: post #40

Earlier quoted context omitted.

Thank you for pointing this out! You are correct that we have not yet decided on a license. I will admit my ignorance on this topic, and I didn't want us to rush into selecting a license that is inappropriate. Which one should we choose?

If it's for the win (?), the most permissible is the one you choose. This is a extraordinarily competitive space. The sooner you make the choice and it's MIT, the sooner I personally put forth serious contribution time and the faster you grow in the broad and competitive ecosystem. Your main options are GNU All-permissive License, MIT License, BSD licenses, Apple Public Source License and Apache license. It is recomm…

Done: https://github.com/OpenAdaptAI/OpenAdapt/blob/main/LICENSE

Re: Fuyu-8B: A multimodal architecture for AI agents

#46

Post by Jackson__ »

fpgaminer wrote: post #19

The architecture is quite compelling. I would not have expected it to work as well as it does. Glancing at the benchmarks it's basically on par with other VLMs in its class, despite having no separate image encoder. Is there an associated paper? Or more specifically, details on the training dataset? It must have been a mix of text and VLM tasks, otherwise one or the other capability would have rotted during training.…

I too would like to know about the training dataset, as I just took a look at the one for LLava[0], and found out that they used a pretty big amount of BLIP auto generated captions.

This seemed a bit surreal to me, like trying to train an LLM with the outputs of a worse performing smaller LLM.

[0] https://github.com/haotian-liu/LLaVA/blob/main/docs/Data.md#...

Re: Fuyu-8B: A multimodal architecture for AI agents

#47

Post by amks »

abrichr wrote: post #33
tasdfqwer0897 wrote: post #2

Hey I work at Adept and helped make this! Happy to answer questions. The thing I think is especially neat/notable is how simple you can make the model architecture while still getting good performance. I expect we'll continue to see bits of these models get deleted in the next few years Note that you can get the model weights on HuggingFace here: https://huggingface.co/adept/fuyu-8b

Thank you for the release! What can you tell us about this: > Our internal models (based on Fuyu) have extra capabilities related to our product. In particular, > 1. They can reliably perform OCR on high-resolution images > 2. They can do fine-grained localization of text and UI elements within those images > 3. They can answer questions about images of UIs Is this just a matter of additional fine tuning, or are ther…

Even with experiments with just adding additional fine-tuning, we've seen models gain these capabilities!

Re: Fuyu-8B: A multimodal architecture for AI agents

#48

Post by MattPalmer1086 »

startupsfail wrote: post #43
outofpaper wrote: post #36

Earlier quoted context omitted.

Model weight are akin to markov chains and compressed data. They are direct representations of the data they where created from in the same way that markov chains are created from hidden markov chains and Zipped files are created from files. Zipping a file does not grant the copyright protection of the zipped output beyond the copyright of the original file. Moreover the American federal registrar has officially stat…

If you take some copyrighted data, a set of books, for example. And count words in these books and then plot a distribution of top 100 word frequencies. The copyright for that new image would belong to you.

Copyright in the specific image sure, but not the graph itself. Someone else could do the same thing and make their own graph image.

Re: Fuyu-8B: A multimodal architecture for AI agents

#49

Post by schleck8 »

mandelken wrote: post #16
JimDabell wrote: post #9

Earlier quoted context omitted.

What’s the situation with the license? Your blog post says you are open sourcing it, but it’s currently only available under a non-commercial license instead. Is an open source release forthcoming?

You can download the weights on Hugginface. I believe the copyright on AI model weights in the US is not fully established, but so far it has been held that a list of numbers can not be copyrighted, so likely the same applies to model weights. Note that you don't have to enter into an agreement with Adept to use the model. Alternatively, use and download the weights in Japan that has explicitly no copyright on AI mod…

I highly doubt that any of this will hold up infront of a court. For intellectual property not just the result is important but also the creation process, and there is enough work going into the data science here

Re: Fuyu-8B: A multimodal architecture for AI agents

#50

Post by schleck8 »

JimDabell wrote: post #9
tasdfqwer0897 wrote: post #2

Hey I work at Adept and helped make this! Happy to answer questions. The thing I think is especially neat/notable is how simple you can make the model architecture while still getting good performance. I expect we'll continue to see bits of these models get deleted in the next few years Note that you can get the model weights on HuggingFace here: https://huggingface.co/adept/fuyu-8b

What’s the situation with the license? Your blog post says you are open sourcing it, but it’s currently only available under a non-commercial license instead. Is an open source release forthcoming?

It's open source by their definition, that is source available (open). Everyone always thinks the term open source is protected in any way while the entity that has established the commercial usage aspect is the Open Source Foundation. And noone is forced to abide by their ideology

FOSS meets the commercial usage requirement much better. Otherwise the term FOSS would be redundant.

Post reply on HN