Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

211–220 of 337 posts

Re: MiniGPT-4

#211

Post by collaborative »

Uehreka wrote: post #209
collaborative wrote: post #153

Earlier quoted context omitted.

At this point the letters GPT make more sense than "AI" or "LLM" in many peoples minds

Hard disagree. Outside of the brand name ChatGPT, lay members of the general public are way more likely to call these chatbots (like Bard and Bing) “AIs” than “GPTs”. And although GPT could technically refer to any model that uses a Generative Pre-trained Transformer approach (although it probably wouldn’t be an open-and-shut case), the mark “GPT-4” definitely is associated with OpenAI and their product, and you can’…

So OpenAI ostensibly owns "GPT4" according to your argument. But does it own "MiniGPT4"? I hope you see the absurdity of this.

Let's not discuss the amount of copyright licenses OpenAI has already infringed, too

Re: MiniGPT-4

#212

Post by therein »

brycedriesenga wrote: post #147
ikurei wrote: post #109

Earlier quoted context omitted.

> This ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy. Just like any discussion between advanced web devs would make any humble woodworker feel? And just like any discussion between advanced woodworkers would make a humble web dev feel? "It's really simple, they're just using a No. 7 jointer plane with a high-angle frog and a PM-V11 blade to flatten those curly birch boards, then a…

Hey, guys. Hey. Ready to talk plate processing and residue transport plate funneling? Why don't we start with joust jambs? Hey, why not? Plates and jousts. Can we couple them? Hell, yeah, we can. Want to know how? Get this. Proprietary to McMillan. Only us. Ready? We fit Donnely nut spacing grip grids and splay-flexed brace columns against beam-fastened derrick husk nuts and girdle plate Jerries, while plate flex tan…

Just tell me do we need a turbo encabulator or not?

Re: MiniGPT-4

#213

Post by rafaelero »

visarga wrote: post #184
sho wrote: post #14

Earlier quoted context omitted.

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

> take BLIP2's ViT-L+Q-former This thing takes an image and creates a representation matrix. > connect it to Vicuna-13B with a linear layer Vicuna is an open LLM, pretty good quality, not as good as GPT3.5 though. This is the beautiful part - a mere multiplication is enough to convert the image tensor to text tensor. One freaking line of code, and a simple one. > and train just the tiny layer on some datasets of imag…

> This is the beautiful part - a mere multiplication is enough to convert the image tensor to text tensor. One freaking line of code, and a simple one.

I thought they were creating image tokens based on the queries during finetuning and appending them to the language model. They are not text tokens.

Re: MiniGPT-4

#214

Post by sigmoid10 »

cs702 wrote: post #115
152334H wrote: post #6

On a technical level, they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs. But the results are pretty amazing. It completely knocks Openflamingo && even the original blip2 models out of the park. And best of all, it arrived before OpenAI's GPT-4 Image Modality did. Real win for Open Sourc…

Indeed, really simple. And yes, the results are shockingly good. But what I find most remarkable about this is that the ViT-L+Q-former's hidden states are related by only a linear projection (plus bias) to the Vicuna-13B's token embeddings: emb_in_vicuna_space = emb_in_qformer_space @ W + B These two models are trained independently of each other, on very different data (RGB images vs integer token ids representing s…

>somehow they learn to embed different data in feature vectors that are so... similar

At it's core, BLIP2 already projects RGB inputs into text token space and Vicuna (or rather LLaMA) uses such tokens as inputs as well as outputs. The only reason why a linear layer is needed at all is because they are not trained at the same time, so you still have to move text embeddings from one space to another. But it should not be surprising at all that one hidden linear layer suffices to do just that (see the universal approximation theorem [1]). This approach is just an efficient way to combine different models for downstream fine-tuning tasks while keeping their weights frozen, but it is neither new nor particularly surprising.

[1] https://en.wikipedia.org/wiki/Universal_approximation_theore...

Re: MiniGPT-4

#215

Post by spyder »

dom96 wrote: post #62

Can this decode CAPTCHAs? Are we finally in a place where CAPTCHAs are useless?

Just tried it with a simple 4 character one and it's bad at it, detects 1 or two characters correctly from the 4 if it outputs anything. It's probably better with the "select the traffic lights" kind of captchas, but those are also already possible to solve with other image models too if I remember correctly.

Re: MiniGPT-4

#216

Post by HansHamster »

Just tried it with a random schematic:

> The image shows a block diagram of a power amplifier circuit with a power supply and a load resistor. The power supply is connected to the input of the amplifier, which is then connected to the output. The load resistor is connected to the output of the amplifier and the power supply. The amplifier is designed to amplify the input signal to a higher level, which is then sent to the load resistor to power the load. The power supply provides the necessary voltage to operate the amplifier.

On the one hand I'm impressed that it recognises an amplifier, but just about everything in the description is (slightly) wrong. It is not a power amplifier, the power supply is not connected to the input (why should it be?), there is no load resistor...

Re: MiniGPT-4

#217

Post by sandkoan »

HarHarVeryFunny wrote: post #192
Tepix wrote: post #181

Earlier quoted context omitted.

> 1T GPT-4 The number of parameters used for GPT-4 is unknown.

I got the 1T GPT-4 number from here - this is the video that goes with the Microsoft "Sparks of AGI" paper, by a Microsoft researcher that had early access to GPT-4 as part of their relationship with OpenAI. https://www.youtube.com/watch?v=qbIk7-JPB2c

Bubeck has clarified that the "1 trillion" number he was throwing around was just a hypothetical metaphorical—it was in no way shape or form implying that GPT-4 has 1 trillion parameters [0].

[0] https://twitter.com/SebastienBubeck/status/16441515797238251...

Re: MiniGPT-4

#218

Post by pc86 »

MattPalmer1086 wrote: post #193
pc86 wrote: post #136

Earlier quoted context omitted.

Maybe it's because of how I use it, but the code ChatGPT gives me has always been super helpful and 99% correct. But, we have a policy at work not to use it for work product so I have to spend time changing enough of it where it's different, and I'm never copy/pasting anything. Enough changes to the structure and variables to make it sufficiently different that it can't be considered pasting company data into GPT, as…

Would you say that this satisfies the spirit of the company policy? Or is it a bit of a hack to get around it? I ask because we are about to produce a similar policy at work. We can see the advantages of it, but likewise, we can't have company data held in their systems.

The policy is to not send any "sensitive company data" into ChatGPT, which I 100% agree with. How we implement a given Vue component or a particular API isn't sensitive or particularly novel so if I strip the business logic out I do honestly believe I'm complying with the spirit of the policy.

Re: MiniGPT-4

#219

Post by cs702 »

sigmoid10 wrote: post #214
cs702 wrote: post #115

Earlier quoted context omitted.

Indeed, really simple. And yes, the results are shockingly good. But what I find most remarkable about this is that the ViT-L+Q-former's hidden states are related by only a linear projection (plus bias) to the Vicuna-13B's token embeddings: emb_in_vicuna_space = emb_in_qformer_space @ W + B These two models are trained independently of each other, on very different data (RGB images vs integer token ids representing s…

>somehow they learn to embed different data in feature vectors that are so... similar At it's core, BLIP2 already projects RGB inputs into text token space and Vicuna (or rather LLaMA) uses such tokens as inputs as well as outputs. The only reason why a linear layer is needed at all is because they are not trained at the same time, so you still have to move text embeddings from one space to another. But it should not…

Thanks. Your comment about BLIP2 already projecting RGB inputs into (a different) text token space makes sense to me. See also fpgaminer's comment at https://news.ycombinator.com/item?id=35603246 . However, I don't see how the universal approximation theorem is relevant here. The fact that deep models with sufficient capacity can approximate any function does not imply that two deep models trained independently of each other on different tasks will learn to approximate functions that relate to each other only by a linear transformation.

Re: MiniGPT-4

#220

Post by sterlind »

nobrains wrote: post #177

I think the real news is where this research is coming from. It is from King Abdullah University of Science and Technology in Saudi. The arab world has recently been quite behind in tech research, but this is good news overall for the arab world, which will inspire other universities and researchers in the region to one-up their colleagues, which is overall good for the entire world.

I would be more thrilled if it were literally any other country in the Arab world, but maybe that will come.
Post reply on HN