Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiniGPT-4

minigpt-4.github.io

Post reply on HN
Print view View original

331–337 of 337 posts

Re: MiniGPT-4

#331

Post by nullsense »

thewarrior wrote: post #182
sho wrote: post #14

Earlier quoted context omitted.

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

I've only been reading ML stuff for a few months and I kind of understand what it's saying. This stuff isn't as complex as its made out to be. It's just a bunch of black boxes AKA "pure functions". BLIP2's ViT-L+Q-former AKA //I give you a picture of a plate of lobster it will say "A plate of lobster". getTextFromImage(image) -> Text Vicuna-13B AKA //I give you a prompt and you return completion ChatGPT style getComp…

This feels like such an accessible explanation.

Re: MiniGPT-4

#332

Post by famouswaffles »

cs702 wrote: post #221
fpgaminer wrote: post #200

Earlier quoted context omitted.

BLIP2 is a contrastive Image-Language model. The embeddings from the BLIP2 image model are already both aligned with text, and linear. It should not be a surprise that only a projection is required to translate it to LLaMA's embedding space.

This is the best answer. It makes sense to me. Thank you :-)

as well as this - https://llava-vl.github.io/, Just found this paper that demonstrated this a few months ago (that somehow language and vision models learn representations similar enough that linear projection is enough) https://arxiv.org/abs/2209.15162

Re: MiniGPT-4

#333

Post by nullsense »

pc86 wrote: post #132
sho wrote: post #14

Earlier quoted context omitted.

> they're doing something really simple -- take BLIP2's ViT-L+Q-former, connect it to Vicuna-13B with a linear layer, and train just the tiny layer on some datasets of image-text pairs Oh yes. Simple! Jesus, this ML stuff makes a humble web dev like myself feel like a dog trying to read Tolstoy.

Big "a monad is just a monoid in the category of endofunctors" vibes from this one.

But that's literally what it is...

Re: MiniGPT-4

#334

Post by cs702 »

famouswaffles wrote: post #332
cs702 wrote: post #221

Earlier quoted context omitted.

This is the best answer. It makes sense to me. Thank you :-)

as well as this - https://llava-vl.github.io/ , Just found this paper that demonstrated this a few months ago (that somehow language and vision models learn representations similar enough that linear projection is enough) https://arxiv.org/abs/2209.15162

Thank you for sharing this. I would not have expected that. It does seem pretty wild.

Re: MiniGPT-4

#335

Post by cs702 »

famouswaffles wrote: post #327
fpgaminer wrote: post #200

Earlier quoted context omitted.

BLIP2 is a contrastive Image-Language model. The embeddings from the BLIP2 image model are already both aligned with text, and linear. It should not be a surprise that only a projection is required to translate it to LLaMA's embedding space.

apparently you can project directly with CLIP. See here - https://llava-vl.github.io/ . This seems pretty wild to me.

That seems pretty wild to me too.

Re: MiniGPT-4

#336

Post by HarHarVeryFunny »

sandkoan wrote: post #217
HarHarVeryFunny wrote: post #192

Earlier quoted context omitted.

I got the 1T GPT-4 number from here - this is the video that goes with the Microsoft "Sparks of AGI" paper, by a Microsoft researcher that had early access to GPT-4 as part of their relationship with OpenAI. https://www.youtube.com/watch?v=qbIk7-JPB2c

Bubeck has clarified that the "1 trillion" number he was throwing around was just a hypothetical metaphorical—it was in no way shape or form implying that GPT-4 has 1 trillion parameters [0]. [0] https://twitter.com/SebastienBubeck/status/16441515797238251...

FWIW, Stephen Gou, Manager of ML at Cohere, is currently doing a Reddit AMA, and is also guessing at 1T params for GPT-4.

https://www.reddit.com/r/IAmA/comments/12rvede/im_stephen_go...

Re: MiniGPT-4

#337

Post by gandolfi »

MacsHeadroom wrote: post #264
tarruda wrote: post #18

Looking forward to the next generation of cheap GPUs with enough VRAM to run models like Vicuna-13 locally.

Vicuna-13B in GPTQ 4bit has almost no perplexity/quality loss and fits in just 8GB of RAM or VRAM. I run it on my phone CPU and get ~4 tokens per second. On my laptop CPU I get 8 tokens per second. On a $200 P40 I run LLaMA-33B at 12 tokens per second in GPTQ 4bit. A consumer 3090 gets over 20 tokens per second for LLaMA-33B and 30 tokens/second for Vicuna-13B.

What cheap card do you advise with Ryzen 5 2400g and Motherboard b450M ? P40, M40, Mi25, 3090 ...? i want to use vicuna 30b smoothly.

thanks

Post reply on HN